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Abstract 


This  paper  presents  a  model  for  measuring  the  benefits  of  improved 
accuracy  in  test  equipment.     The  model  permits  calculation  of  the  probability 
of  accepting  a  unit  under  test  (UUT)  that  is  out  of  specifications  (i.e., 
Consumer's  Loss)  and  the  probability  of  rejecting  a  UUT  that  is  within 
specifications  (i.e.,  Producer's  Loss)  for  alternative  levels  of  test 
equipment  accuracy.    Accuracy  is  defined  in  terms  of  both  the  systematic  and 
the  random  measurement  error  of  the  equipment.     Other  parameters  that  are 
taken  into  account  by  the  model  are  the  mean  and  variance  of  the  UUT  attribute 
of  interest,  the  performance  specifications  of  the  UUT,  and  the  test 
specifications  which  define  acceptance  and  rejection  in  terms  of  test 
measurement  results.    A  discussion  of  the  economic  consequences  of  Consumer's 
Loss  and  Producer's  Loss  is  included.    The  model  may  be  used  to  optimize  both 
procurement  policy  for  new  test  equipment  as  well  as  maintenance  and 
calibration  policy  for  existing  test  equipment. 

Key  words:     accuracy;  automatic  test  equipment;  calibration;  Consumer's  Loss; 

economics;  error  analysis;  measurement;  Producer's  Loss;  quality 
assurance;  random  error;  systematic  error;  testing. 
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1.  Introduction 

1.1    Background  and  Objective 

This  report  presents  a  model  for  measuring  the  benefits  of  improved 
accuracy  in  test  equipment.    Although  the  model  could  be  applied  to  many  types 
of  test  equipment,  the  focus  of  this  report  is  on  that  broad  category  of 
electronic  test  equipment  commonly  referred  to  as  Automatic  Test  Equipment 
(ATE).     For  purposes  of  this  discussion,  ATE  is  defined  as  electronic  test 
equipment  that  is  controlled  by  microprocessors  or  programmable  computers  and 
is  capable  of  high  speed  testing  of  individual  electronic  components,  printed 
circuit  boards,  wiring  harnesses,  or  entire  electronic  system  assemblies.! 

The  origin  of  expected  improvements  in  the  accuracy  of  ATE  lies  in  a 
proposal  by  the  National  Bureau  of  Standards  (NBS)  Center  for  Electronics  and 
Electrical  Engineering  (CEEE)  to  develop  prototypes  for  and  support  the 
implementation  of  a  new  calibration  system  that  satisfies  the  special 
requirements  of  ATE.     This  proposed  system,  called  Dynamic  Transport  Standards 
(DTS),  would  address  two  shortcomings  of  the  current  approach:     "...(1)  the 
lack  of  adequate  ATE  calibration  in  its  operational  environment  under 
operational  conditions  at  the  UUT  [unit  under  test]  terminals  and  (2)  the 
extensive  range  extension  and  bootstrapping  activities  in  the  building  up  of 
multi-parameter  multi-level  standards  in  numerous  standards  and  calibration 


This  definition  is  adapted  from  an  unsigned  article,  "Electronic  Component 
Testing  with  ATE"  (Part  I),  Quality,  Vol.  20,  No.  10  (October  1981), 
pp.  16-20. 
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laboratories  in  order  to  achieve  traceability  to  national  standards  at  NBS."1 
Two  types  of  improvements  related  to  the  performance  of  ATE  could  result 
from  the  use  of  the  DTS  system  of  calibration,  once  it  is  fully  implemented. 
These  potential  improvements  would  both  be  in  the  form  of  reductions  in  the 
overall  measurement  error  of  ATE.     The  first  improvement  is  a  decrease  in  the 
systematic  error  or  bias  present  in  the  test  equipment  measurements.  The 
second  type  of  improvement  possible  in  the  accuracy  of  ATE  involves  a  decrease 
in  the  random  measurement  error  of  the  test  results.     This  second  type  of 
improvement  is  equivalent  to  an  increase  in  measurement  precision.  The 
benefits  of  both  these  types  of  improved  accuracy  (reductions  in  measurement 
error)  are  determined  by  the  model  presented  in  this  report. 

1.2  Scope 

The  model  is  stochastic  in  the  sense  that  it  is  based  on  two  probability 
distributions  and  the  measures  of  the  benefits  from  improved  accuracy  are 
stated  in  terms  of  probabilities.     The  probability  distributions  are  those  of 
the  UUT  performance^  being  tested  and  of  the  measurement  error  of  the  test 
equipment.     The  model  can  be  used  to  analyze  accuracy  improvements  for 
practically  any  application  of  test  equipment  as  long  as  these  two  probability 


Barry  A.  Bell  and  Oskars  Petersons,  "ATE  Calibration  by  Means  of  Dynamic 
Transport  Standards,"  Proceedings  of  Autotestcon  81:   International  Automatic 
Testing  Conference  (Orlando,  FL,  October  19-21,   1981),  Institute  of 
Electrical  and  Electronic  Engineers  (IEEE  Catalogue  No.  81  CH  1716-0),  1981, 
pp. 280-287. 

UUT  performance  refers  to  the  value  of  the  particular  UUT  attribute  that  is 
being  measured  by  the  test  equipment. 
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distributions  can  be  assumed  to  be  normal.     That  is,  the  attribute  of  the  UUT 
that  is  being  tested  as  well  as  the  test  equipment  measurement  error  must  both 
be  approximately  normally  distributed. 

The  measures  of  benefits  determined  by  the  model  are  based  on  reductions 
in  the  probabilities  of  the  two  types  of  errors  possible  with  every  test 
situation:     Consumer's  Loss  (CL)  and  Producer's  Loss  (PL).*    CL  is  the 
probability  of  accepting  a  UUT  that  does  not  conform  to  the  performance 
specifications  for  the  attribute  being  tested.    PL  is  the  probability  of 
rejecting  a  UUT  that  does  in  fact  conform  to  the  performance  specifications. 
The  terms  "consumer"  and  "producer"  are  used  to  indicate  which  interest  group 
would  typically  bear  the  cost  of  each  error  type  in  a  production  testing 
context.    During  product  warrantee  periods,  however,  most  of  the  cost  arising 
from  CL  would  typically  be  borne  by  producers.    Moreover,  when  testing  is 
being  conducted  as  part  of  a  maintenance  function,  the  costs  of  both  CL  and  PL 
usually  remain  within  the  organization. 

The  model  expresses  both  probabilities,  CL  and  PL,  as  explicit  functions 
of  measurement  bias  (systematic  error) ,  represented  by  the  mean  of  measurement 
error,  and  of  measurement  imprecision,  represented  by  the  standard  deviation 
of  measurement  error.    These  functions  can  be  used  to  calculate  directly  the 
benefits,  as  measured  by  lower  probabilities  of  errors,  from  reduced  measure- 
ment bias  and/or  imprecision.  When  applied  to  a  case  study,  an  economic  value, 
in  terms  of  dollars  saved  per  time  period,  can  be  established  for  these 


1 These  probabilities  are  also  referred  to  as  Consumer's  Risk  and  Producer's 
Risk,  respectively. 
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reduced  probabilities.    The  economic  value  depends  on  the  number  of  tests 
conducted  in  a  time  period  and  on  the  particular  test  situation  being 
analyzed.     Multiplication  of  each  probability  reduction  by  the  number  of  items 
to  be  tested  gives  the  expected  reduction  in  the  number  of  occurrences  of  each 
type  of  test  error.    The  economic  value  of  each  reduced  error  occurrence  can 
then  be  determined  by  identifying  and  categorizing  the  cost  consequences  of  a 
single  CL  and  PL  occurrence.     Separate  estimates  of  the  value  of  each  cost 
consequence  have  to  be  derived.     Examples  of  such  cost  consequences  for  a  CL 
occurrence  are  field  repair  costs,  accidents  due  to  equipment  failure,  and  for 
military  applications  the  possible  effects  on  readiness  and  mission  success 
rates.    Examples  of  cost  implications  for  a  PL  occurrence  are  unnecessary 
diagnostic  and  repair  costs,  the  costs  of  unnecessary  retesting  (including 
packing  and  shipping  to  higher  echelon  laboratories) ,  excessive  inventory  of 
parts  and  UUTs  because  of  downtime,  and  reduced  production  yields. 

Because  testing  is  conducted  for  a  variety  of  purposes,  the  applicability 
as  well  as  the  dollar  value  of  each  of  these  cost  implications  (among  others 
that  may  be  identified)  can  vary  greatly  depending  on  the  particular  test 
situation  being  analyzed.    Examples  of  the  many  test  applications  possible  are 
production  testing  both  at  intermediate  stages  and  at  the  final  product  stage 
(i.e.,  outgoing  tests),  incoming  testing  of  components  to  be  used  in 
manufacturing,  and  maintenance  testing.    The  economic  model  presented  in  this 
report  can  be  used  to  analyze  all  of  these  test  applications,  provided  the 
basic  nature  of  the  measurement  process  is  analog  rather  than  digital. 
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That  is,  the  principal  characteristic  of  the  UUT  that  is  being  measured  must 
be  a  continuous  variable  rather  than  a  binomial  or  other  type  of  discrete 
variable.     This  requirement  is  not  too  restrictive,  however,  because  even  when 
semiconductor  devices  that  are  primarily  digital  in  design  are  being  tested, 
some  critical  analog  measurements  must  be  made. 

This  economic  model  may  be  employed  to  guide  managers  in  making 
cost-effective  decisions  regarding  research  planning,  maintenance  and 
procurement  policy,  and  product  quality  assurance  functions.     For  example,  by 
determining  the  effects  of  alternative  calibration  support  strategies  on  CL 
and  PL,  the  model  can  assist  program  managers  in  selecting  areas  of 
calibration  research  that  have  the  highest  potential  payoff.     In  the  area  of 
maintenance  and  procurement  policy,  the  model  can  be  used  to  optimize  the 
frequency  and  quality  of  calibration  procedures  with  existing  test  equipment, 
as  well  as  to  select  the  best  new  test  equipment  through  a  comparison  of 
additional  costs  with  the  economic  benefits  of  expected  accuracy  improvements. 
Moreover,  in  the  area  of  quality  assurance,  the  model  can  be  applied  to 
determine,  for  a  particular  product  and  test  environment,  which  test 
specifications  optimize  the  economic  trade-off  between  CL  and  PL. 

1.3  Organization 

The  next  section  describes  the  mathematical  formulation  of  the  model  for 
calculating  CL  and  PL  as  functions  of  the  mean  and  the  standard  deviation  of 
the  measurement  error.     Section  3  discusses  the  results  of  applying  the  model 
to  an  analysis  of  how  changes  in  the  mean  and  standard  deviation  of  the 
measurement  error  can  be  expected  to  affect  CL  and  PL.     Section  4 
summarizes  the  results  and  describes  several  useful  applications  of  the  model. 
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Appendix  A  gives  complete  documentation  for  BINORMAL.CL-PL,  the  interactive 
computer  program  developed  to  calculate  CL  and  PL  for  a  wide  variety  of  test 
situations.    Appendix  B  provides  plots  of  CL  and  PL  as  functions  of  the  mean 
and  standard  deviation  of  measurement  error,  and  illustrates  how  the  plots  can 
be  used  to  derive  estimates  of  CL  and  PL  for  certain  cases. 
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2.  Mathematical  Formulation  of  the  Model 

This  section  begins  by  summarizing  the  work  of  statisticians  who 
developed  the  basic  model  for  computing  CL  and  PL  in  the  context  of  100 
percent  testing.    Then  this  basic  model  is  extended  to  incorporate  the  effect 
of  possible  changes  in  the  accuracy  of  the  test  equipment.     These  accuracy 
changes  are  mathematically  specified  both  in  terms  of  uncorrected  measurement 
bias  and  random  measurement  error.    The  notation  that  will  be  used  in  the 
model  is  summarized  in  table  2.1. 
2.1    The  Basic  Model 

Three  key  papers  published  in  the  1950' s  describe  how  to  use  CL  and  PL 
when  selecting  appropriate  test  specification  limits.     Eagle  defined  the  basic 
concepts  of  CL  and  PL  and  published  plots  of  their  values  as  functions  of  the 
test  specification  limits  and  the  random  measurement  error  for  a  fixed 
performance  specif ication. 1    Eagle's  discussion  was  restricted  to  the  case  in 
which  both  the  upper  and  lower  test  specification  limits  and  the  performance 
specification  limits  are  symmetric  with  respect  to  the  mean  of  the  performance 
distribution.     In  an  article  published  in  the  same  issue  of  Industrial  Quality 
Control ,  Grubbs  and  Coon  maintained  this  symmetry  restriction  while  applying 
the  model  to  optimize  the  decision  of  how  to  set  the  test  specification  in 
relation  to  the  performance  specif ications. 2    They  developed  decision  rules 


Alan  R.  Eagle,   "A  Method  for  Handling  Errors  in  Testing  and  Measuring," 
Industrial  Quality  Control,  Vol.  10,  No.  5  (March  1954),  pp  10-15. 

Frank  E.  Grubbs  and  Helen  J.  Coon,   "On  Setting  Test  Limits  Relative  to 
Specification  Limits,"  Industrial  Quality  Control,  Vol.   10,  No.  5  (March 
1954),  pp  15-20. 
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Table  2.1     Notation  Used  in  the  Model  and  in  Appendix  B. 


Symbol  Definition 


UUT  Unit  Under  Test:     product  or  product  component  that  is 

being  tested. 

CL  Consumer's  Loss:     the  probability  of  accepting  a  UUT  that 

does  not  conform  to  the  performance  specifications  for 
the  attribute  being  tested. 

PL  Producer's  Loss:     the  probability  of  rejecting  a  UUT 

that  does  in  fact  conform  to  the  performance 
specifications  for  the  attribute  being  tested. 

u  Performance  values  of  the  particular  UUT  attribute  that 

is  being  tested. 

e  Error  values  that  occur  when  u  is  measured  with  the  test 

equipment . 

in  Observed  measurement  values  of  u  as  taken  by  the  test 

equipment ,  m  =  u  +  e . 

p  Statistical  correlation  between  u  and  m. 

pu  Mean  of  the  UUT  performance  distribution. 

au  Standard  deviation  of  the  UUT  performance  distribution. 

ue  Mean  of  the  measurement  error  distribution,  also  known  as 

undetected  systematic  measurement  error  or  bias. 

oe  Standard  deviation  of  the  measurement  error  distribution, 

also  known  as  measurement  imprecision. 

S|j  Upper  specification  limit  for  UUT  performance. 

Sl  Lower  specification  limit  for  UUT  performance. 

TA  Tolerance  adjustment  of  the  test  equipment:  the 

difference  between  the  performance  specification  limits 
and  the  test  specification  limits.     In  this  discussion, 
TA  is  assumed  to  be  equal  for  upper  and  lower 
specifications.     TA  is  an  adjustment  typically  made  to 
allow  for  measurement  error  in  the  test  equipment.  A 
positive  TA  value  means  that  the  test  specifications 
lie  inside  the  performance  specifications.     Thus,  a 
positive  TA  makes  the  test  specifications  more 
stringent  (i.e,  more  likely  to  reject  a  UUT)  than  the 
unadjusted  performance  specifications. 
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Table  2.1     Notation  Used  in  the  Model  and  in  Appendix  B  (Cont'd) 


Symbol 

Definition 

Su-TA 

Upper  test  specification  limit. 

SL+TA 

Lower  test  specification  limit. 

M 

Standardized  upper  performance  specification  limit. 

"•2 

Negative  of  the  standardized  lower  Derf ormance 

It              t»  L>  JL-  V               \-/  J-          fcr  1  A  V**               fc-  t-t  LAVA  CA  A.  VA  A  u        \-l         -L.  K>  W  V.  A.          L/  t_  A.  J.  V-/  A>               A  A  V-  ^ — 

specification  limit. 

Standardized  upper  test  limit. 

<12 

Negative  of  the  standardized  lower  test  limit. 
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for  setting  the  test  specification  limits  in  order  to  optimize  three 
alternative  objective  functions:     (1)  to  make  CL  and  PL  equal;   (2)  to  minimize 
the  sum  of  CL  and  PL;  and  (3)  to  minimize  the  weighted  sum  of  CL  and  PL,  where 
the  weights  could  be  interpreted  as  the  relative  costs  of  a  single  occurrence 
of  each  type  of  error.     Five  years  later,  Owen  and  Wiesen  published  a  method 
of  calculating  CL  and  PL  using  graphs  and  extended  the  basic  model  to  include 
the  analysis  of  one-tail  tests  as  well  as  of  asymmetric  settings  on  both  the 
test  specification  limits  and  the  performance  specification  limits  for 
two-tail  tests. *     (A  two-tail  test  has  both  upper  and  lower  test  specification 
limits,  while  a  one-tail  test  has  only  one  of  these  limits.)     For  these 
additional  cases,  Owen  and  Wiesen  develop  optimal  decision  rules  for  setting 
test  limits  using  the  three  objective  functions  suggested  by  Grubbs  and  Coon. 
In  addition,  they  develop  test  limit  decision  rules  based  on  an  interesting 
objective  function  suggested  by  Tingey  and  Merrill, 2  namely  to  minimize  the 
sum  of  the  costs  of  CL  and  PL  when  the  cost  per  occurrence  of  CL  varies  in 
proportion  to  the  degree  of  nonconformance  of  the  accepted  product  (i.e.,  how 
far  out  of  specifications  the  product  is). 

The  general  test  situation  being  analyzed  in  all  of  the  above  modeling 
efforts  is  that  in  which  a  single  measurement  is  taken  on  a  continuously 
variable  characteristic  of  every  UUT  in  the  process  being  monitored. 


D.  B.  Owen  and  J.  M.  Wiesen,  "A  Method  of  Computing  Bivariate  Normal 
Probabilities:     With  an  Application  to  Handling  Errors  in  Testing  and 
Measuring,"  The  Bell  System  Technical  Journal,  Vol.  38  (March  1959), 
pp.  553-572. 

F.  H.  Tingey  and  J.  A.  Merrill,  "Minimum  Risk  Specification  Limits,"  AEC 
Research  and  Development  Report  IDO-16396  (Phillips  Petroleum  Co.,  Atomic 
Energy  Division,  Contract  No.  AT  (10-l)-205),  July  30,  1957. 
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As  indicated  in  the  previous  section,  the  process  may  be  manufacturing, 
screening  of  input  components  prior  to  assembly,  or  periodic  maintenance.  The 
test  equipment  with  which  the  measurements  are  taken  is  subject  to  random 
error.     Because  of  the  presence  of  measurement  error  there  is  a  nonzero 
probability  that  bad  units  will  be  accepted  (i.e.,  CL)  ,  and  similarly  a 
nonzero  probability  that  good  units  will  be  rejected  (i.e.,  PL).     The  basic 
model  formulated  in  the  above  papers  assumes  that  the  distribution  of  the  UUT 
performance  characteristic  being  measured  is  normal  (Gaussian)  with  mean  uu 
and  standard  deviation  ou.     It  is  further  assumed  that  the  distribution  of 
the  test  measurement  error  is  normal  with  a  mean  of  zero  and  standard 
deviation  o£.l     The  upper  and  lower  UUT  performance  specification  limits  are 
based  on  engineering  considerations  and  assumed  to  be  set  at  Sjj  and  Sl  , 
respectively. 2    To  adjust  for  the  random  error  known  to  be  present  in  the 
test  equipment  measurements,  the  test  specification  limits  are  generally 
assumed  to  be  set  at  values  different  from  Sy  and  Sl.     The  magnitude  and 
direction  of  this  adjustment  is  given  by  TA,  the  tolerance  adjustment  of  the 
test  equipment. 3    Thus,  the  upper  test  specification  limit  is  set  at  Sjj-TA 
and  the  lower  limit  is  set  at  SL+TA.     Thus,  for  TA>0  the  test  limits  are 
placed  inside  the  UUT  performance  specification  limits,  which  makes  the  test 


In  the  formulation  of  Owen  and  Wiesen,  the  mean  of  the  test  measurement  error 
distribution  can  assume  nonzero  values.     However,  this  bias  is  generally 
assumed  to  be  known  and  corrected  for  in  setting  the  test  specification 
limits . 

This  discussion  of  the  basic  model  and  its  extensions  focuses  on  two-tail 
tests.     The  computer  program  described  in  appendix  A  also  covers  upper  and 
lower  one-tail  tests. 

This  formulation  assumes  that  the  tolerance  adjustment  is  equal  for  both 
tails.     The  computer  program  in  appendix  A  permits  unequal  tolerance 
adjustments . 
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limits  more  stringent  (i.e.,  more  likely  to  reject  UUTs)  than  if  there  were  no 
tolerance  adjustment.     If  a  quality  assurance  manager  considered  measurement 
errors  of  a  particular  test  application  to  be  insignificant  relative  to  the 
variance  in  the  UUT  performance  distribution,  TA  would  typically  be  set  at 
zero.     In  this  case  the  test  limits  would  equal  the  performance 
specifications.    When  measurement  errors  cannot  be  ignored,  a  nonzero  value 
for  TA  is  needed.     For  a  given  test  equipment  precision,  the  value  assigned  to 
TA  should  be  determined  by  the  relative  cost  consequences  of  each  occurrence 
of  CL  and  PL.     The  more  costly  is  an  occurrence  of  CL,  the  greater  TA  should 
be  in  order  to  make  the  test  more  stringent  and  less  likely  to  accept  bad 
UUTs.     Conversely,  the  more  costly  is  an  occurrence  of  PL  (relative  to  CL) , 
the  smaller  (perhaps  even  negative)  TA  should  be  in  order  to  make  the  test 
less  stringent  and  less  likely  to  reject  good  UUTs.     Thus,  when  the  precision 
of  test  equipment  is  given,  the  economic  trade-off  between  CL  and  PL  is  to  be 
balanced  by  setting  the  test  limits  with  respect  to  the  performance 
specifications,  that  is,  by  choosing  the  appropriate  value  for  TA.  Indeed, 
the  primary  motivation  of  this  basic  model,  as  developed  in  the  1950' s,  is  to 
provide  a  rational  basis  for  setting  test  limits. * 

The  formulas  for  calculating  CL  and  PL  are  based  on  the  bivariate  normal 
probability  distribution  and  have  been  worked  out  by  Owen  and  Wiesen.2 
The  performance  values  of  the  particular  UUT  attribute  that  is  being 
measured  and  tested  are  denoted  by  u.     These  values  are  measured  subject  to 
error,  e,  so  that  the  observed  measurement  as  taken  by  the  test  equipment  is 

xThe  very  title  of  the  article  by  Grubbs  and  Coon  is,  "On  Setting  Test  Limits 
Relative  to  Specification  Limits."     In  the  Owen  and  Wiesen  article,  the  major 
concluding  section  is  entitled,  "Special  Criteria  for  Determining 
Specification  Limits." 

^See  section  III,  "Errors  in  Testing  and  Measuring,"  in  Owen  and  Wiesen, 
pp.  562-569. 
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given  by  m  =  u  +  e .    As  long  as  u  and  e  are  independent,  the  correlation 
between  u  and  m  is  given  by  p  =  ou(ou2  +  oe2)~0*5.     Based  on  the  conceptual 
definitions  of  CL  and  PL,  the  following  expressions  are  derived  in  Owen  and 
Wiesen: 

CL  =  LCkj,  -  q2,  p)  -  L(kj,  qx  ,  p)  -  L(k2,  q2  ,  P)  +  L(k2,  -qi  ,  p),  (2.1) 
PL  =  CL  +  G(kx)  +  G(k2)  -  G(qx)  -  G(q2),  (2.2) 

where  L(k,q,p)  is  the  upper  tail  probability  for  the  standardized  bivariate 

normal  distribution, 

(2tt)-1(1-P2)"0*5  /  /  exp  [(2pum-u2-m2)/2(l-p2)]  dm  du,  (2.3) 
k  q 

and  G(k)  is  the  standardized  univariate  normal  cumulative  distribution 
function, 

k 

(2lT)-0.5  J    exp  [-u2/2]  du.  (2. A) 

—00 

The  arguments,  kj  and  k2 ,  of  the  L  and  G  functions  are  the  standardized  upper 
performance  specification  limit  and  the  negative  of  the  standardized  lower 
performance  specification  limit,  respectively,  and  are  defined  as  follows: 

kl  =  (sir^u)/°u>  and 

(2.5) 

k2  =  -(SL-yu)/au 

The  arguments,  qj  and  q2 ,  are  the  standardized  upper  test  limit  and  the 
negative  of  the  standardized  lower  test  limit,  respectively,  and  are  defined 
as  follows: 
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qx  =  (Su  -  TA  -  mu)  (au2  +  ae2)-0.5>  and 

(2.6) 

Q2  =  "CSL  +  TA  -  mu)  (ou2  +  oe2)-0.5 

The  correctness  of  the  formula  for  CL  given  in  expression  (2.1)  can  be 

demonstrated  by  examining  each  of  its  four  terms  (i.e.,  the  L  functions). 

Recall  the  basic  definition  of  CL  as  the  probability  of  accepting  a  bad  UUT. 

The  first  two  terms  in  expression  (2.1)  address  only  those  bad  UUTs  that 

exceed  the  upper  performance  specification,  S^.     The  first  term,  L(kj,  -q£ ,  p), 

is  interpreted  as  the  joint  probability  that  the  true  performance  value  of 

a  UUT  being  tested  exceeds  the  upper  performance  specification  (i.e.,  the  UUT 

is  bad  on  the  high  side),  and  that  the  test  result  is  greater  than  the  lower 

test  specification  (i.e.,  the  UUT  is  not  rejected  by  falling  below  the  lower 

test  specification).     Since  it  is  highly  improbable  for  a  UUT  that  exceeds  Sg 

to  have  a  test  result  falling  below  the  lower  test  limit,  this  first  term 

represents  virtually  all  of  the  UUTs  whose  true  performance  values  are  bad  on 

the  high  side.     The  second  term  of  CL,  L(kj,  qj ,  p)  is  the  joint  probability 

that  a  UUT  exceeds  the  upper  performance  specification  (i.e.,  a  bad  UUT),  and 

that  the  test  result  exceeds  the  upper  test  limit  (i.e.,  the  UUT  is  rejected 

by  falling  above  the  acceptable  limit).     In  other  words,  the  second  terra  in 

expression  (2.1)  can  be  interpreted  as  the  joint  probability  of  a  UUT  that  is 

bad  on  the  high  side  being  correctly  rejected  by  exceeding  the  upper  test 

limit.     Thus  the  difference  between  the  first  two  terms  of  CL, 

L(kj,  -q2 ,  p)  -  L(kj ,  qj ,  p),  represents  the  probability  that  a  UUT  that  is 

bad  on  the  high  side  will  pass  the  test  and  be  accepted.     Thus,  this  difference 

represents  the  upper  tail  portion  of  CL.     Similarly,  the  remaining  two  terms 

in  the  CL  formula  represent  the  probability  a  UUT  that  is  bad  on  the  low  side 

will  pass  the  test  and  be  accepted.     Thus  the  difference  given  by  the  third 

and  fourth  terms  of  expression  (2.1),  -L(k2,  q2  ,  P)  +  L(k2,  ~qi,  p), 

represents  the  lower  tail  portion  of  CL. 
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2.2    Extensions  of  the  Model 

In  this  subsection  the  basic  model  discussed  above  is  extended  to  permit 
the  analysis  of  the  effects  of  changes  in  test  equipment  accuracy  on  CL  and 
PL.     The  elements  of  accuracy  to  be  analyzed  include  both  uncorrected 
measurement  bias  and  random  measurement  error. 

If  a  systematic  error  or  bias  is  present  in  a  measurement  system  and  its 
magnitude  and  direction  are  exactly  known,  then  the  value  of  TA  can  be 
adjusted  accordingly.     This  type  of  known  and  corrected  bias  is  not  of  much 
interest  because  it  has  no  effect  on  CL  and  PL.     On  the  other  hand,  a 
measurement  bias  which  is  unknown  and  uncorrected  will  affect  both  CL  and  PL 
because  such  a  bias  would  result  in  different  values  for  qj  and  q2  in  the 
formulas . 

Denote  ue  to  be  the  undetected  systematic  measurement  error  (bias)  in  the 
test  equipment.     Because  of  this  bias,  the  mean  of  the  observed  measurement 
now  becomes  um  =  \iu  +  ye .     However,  the  standard  deviation  of  m  is 
unaffected  by  the  bias  as  long  as  the  error  and  the  performance  value  are 
independent.     That  is,  am  =  (au2  +  ae 2)0.5 f  as  was  the  case  in  the  absence  of 
bias.     Thus,  p,  which  is  the  ratio  of  au  to  om,  is  also  unaffected  by  the 
introduction  of  the  bias.* 

The  basic  model  must  be  modified  to  take  into  account  explicitly  the 
presence  of  undetected  measurement  bias  with  its  consequent  change  in  the  mean 
of  the  observed  measurement,  um.     The  arguments,  kj  and  k2 ,  given  in  expres- 
sion (2.5)  were  derived  by  standardizing  the  upper  and  lower  performance 
specification  limits  and  taking  the  negative  of  the  lower  standardized  limit. 
The  procedure  for  standardizing  involves  subtracting  the  mean  and  dividing  by 
the  standard  deviation  of  the  relevant  distribution.     Since  kj  and  k2  represent 

i0wen  and  Wiesen,  p.  562. 
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limits  on  the  UUT  distribution,  standardizing  involves  subtracting  yu  and 
dividing  by  au.     The  introduction  of  undetected  measurement  bias  has  no  effect 
on  either  of  these  parameters  of  the  UUT  distribution,  so  that  the  arguments, 
kj  and  k£ ,  are  unaffected  by  such  bias.     Consequently,  the  definitions  of  kj 
and  k2  found  in  expression  (2.5)  are  to  be  used  in  both  the  basic  model  (with 
no  measurement  bias)  and  the  extended  model  (with  measurement  bias). 

In  contrast,  the  introduction  of  undetected  measurement  bias  has  a 
decided  effect  on  the  arguments,  qj  and  q2 ,  defined  for  the  basic  model  in 
expression  (2.6).     These  arguments  represent  limits  on  the  observed 
measurement  distribution  (i.e.,  the  distribution  of  m) .  Consequently, 
standardizing  the  test  limits,  Sy+TA  and  SL~TA,  involves  subtracting  the  mean 
of  m,  which  has  changed  from  yu  in  the  basic  model  to  Uu+jJe  in  the  extended 
model  because  of  the  presence  of  nonzero  measurement  bias  (pe).    As  noted  above, 
the  standard  deviation  of  the  observed  measurement,  om,  remains  unaffected  by 
the  measurement  bias.     Standardizing  the  upper  and  lower  test  limits  using  the 
new  value  of  \im  and  taking  the  negative  of  the  standardized  lower  test  limit 
result  in  the  following  new  definitions  of  the  arguments  q^  and  q2  to  be  used 
in  the  extended  model: 

2  2     a  c 

qj  =  (Sy  -  TA  -  uu  -  ye)(au    +  ae  )~°«5,  and 

(2.7) 

Q2  =  "(SL  +  TA  -  yu  -  pe)(au    +  oe  )-0'5  . 

In  summary,  the  introduction  of  an  undetected  measurement  bias  has  the 
effect  of  reducing  qj  by  an  amount  equal  to  ye(ou2  +  oe2)""0*5  and  of 
increasing  q2  by  the  sarae  amount. *     This  explicit  introduction  of  the  bias, 

AOwen  and  Wiesen  (p.  567)  suggest  that  an  undetected  bias  should  be  embodied 
in  the  model  as  a  change  in  the  UUT  performance  specification  limits,  but 
clearly  the  test  bias  only  affects  the  observed  measurement  result,  m,  and 
leaves  the  underlying  UUT  distribution  and  performance  limits  unchanged. 
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ye ,  constitutes  a  generalization  of  the  basic  model.     The  basic  model  is  now 
seen  as  a  special  case  of  the  more  general,  extended  model.     The  basic  model 
represents  the  case  in  which  ue  ,  the  undetected  measurement  bias,   is  zero. 
Consequently,  the  definitions  of  qj  and  q2  in  expression  (2.7)  can  be  used  in 
all  cases.     This  generalization  of  the  model  permits  direct  calculation  of  the 
effect  of  any  given  undetected  bias  on  CL  and  PL.     In  addition,  this  extended 
model  can  be  used  to  measure  the  effect  of  changing  (i.e.,  increasing, 
reducing,  or  even  eliminating)  an  undetected  bias  by  any  amount.     In  this  way, 
the  economic  effects  of  any  improvements  in  test  equipment  accuracy  related  to 
systematic  measurement  error  can  be  determined.     The  origin  of  such  bias- 
related  accuracy  improvements  could  be  better  equipment,  better  operation  and 
maintenance  personnel  or  procedures,  or  an  improved  system  of  calibration,  such 
as  is  expected  if  the  DTS  system  is  implemented  for  ATE. 

If  no  measurement  bias  is  present,  the  basic  model  could  be  used  to 
analyze  the  effects  of  imprecision  (i.e.,  o£).    On  the  other  hand, 
if  changes  of  test  equipment  precision  are  to  be  analyzed  in  the  presence  of 
bias ,  then  the  extended  model  that  explicitly  incorporates  a  term  for 
measurement  bias  in  the  formulas  for  CL  and  PL  must  be  used .     Indeed ,  both 
analyses  can  be  conducted  using  the  extended  model ,  because  the  bias  term  ue 
can  easily  be  set  equal  to  zero  in  the  q^  and  q2  formulas. 

In  the  CL  and  PL  formulas  of  the  extended  model,  o£  appears  as  part  of  the 
denominators  of  qj ,  q£  and  p.     Thus,  the  determination  of  the  effect  on  CL  and 
PL  of  an  improvement  in  precision  (i.e.,  a  reduction  in  oe )  is  accomplished  by 
first  computing  CL  and  PL  using  the  original  (lower  precision)  value  of  ae  and 
then  recomputing  CL  and  PL  using  the  improved  precision  value  of  oe .  The 
difference  between  the  two  computations  represents  the  reduction  in  the 
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probabilities  of  occurrence  of  the  CL  and  PL  errors  that  could  result  from 
improving  the  precision  (lowering  a£)  of  the  test  equipment.     The  improved 
precision  could  be  realized  by  (1)  more  advanced  test  equipment,  (2)  better 
operation  and  maintenance  procedures  that  reduce  the  random  measurement  error 
arising  from  such  sources  as  electromagnetic  interference,  temperature, 
humidity,  and  pressure,  or  (3)  better  calibration  support  systems. 
Conversely,  the  model  could  determine  the  cost  consequences,  in  terms  of 
increased  values  of  CL  and  PL,  of  using  less  precise  test  equipment. 1 

This  concludes  the  discussion  of  the  basic  model  and  the  extended  version 
of  the  model  which  permits  the  analysis  of  the  effects  on  CL  and  PL  of  changes 
in  the  systematic  and/or  random  measurement  errors  of  test  equipment.  The 
next  section  summarizes  the  results  of  applying  the  extended  model  to 
determine  the  magnitude,  direction,  and  general  pattern  of  these  effects  on  CL 
and  PL  for  a  broad  range  of  test  situations. 


There  are  some  circumstances  in  which  reducing  the  measurement  bias  could 
actually  lead  to  increases  in  CL  and  PL.     Similarly,  it  is  possible  that  a 
decrease  in  ae  may  result  in  an  increase  in  CL  or  PL.     These  cases  are 
discussed  in  section  3. 
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3.  Results 

An  interactive  computer  program  has  been  developed  which  calculates  the 
values  of  the  probabilities,  CL  and  PL,  as  defined  in  the  extended  model 
described  in  section  2.    Complete  documentation  and  a  listing  for  this 
program,  B I NORMAL. CL- PL,  are  provided  in  appendix  A  of  this  report.  The 
program  was  used  to  calculate  CL  and  PL  for  a  wide  range  of  assumed  model 
parameter  values.    The  objective  of  this  section  is  to  use  the  results  of 
these  calculations  to  indicate  how  changes  in  systematic  measurement  error 
(ue)  and  in  the  standard  deviation  of  measurement  error  (oe )  affect  CL  and  PL. 
The  results  reported  here  are  all  based  on  the  case  of  two-tail  tests  with 
performance  specifications  and  test  limits  both  symmetric  around  the  mean 
performance,  uu.     So  that  the  results  can  be  easily  generalized  to  other 
cases,  the  performance  distribution  is  standardized  (i.e.,  Pu=0,  and  ou=l). 
3.1     Effects  of  Systematic  Measurement  Error 

The  analysis  of  the  effects  of  systematic  measurement  error  on  CL  and  PL 
is  conducted  on  two  levels.    The  first  level  of  analysis  focuses  on  the 
direction  of  the  effects  of  changing  systematic  error  on  CL  and  PL.  The 
question  is  whether  an  increase  in  systematic  measurement  error  would  increase 
or  decrease  CL  and  PL.     The  second  level  of  analysis  concerns  the  general 
pattern  and  order  of  magnitude  of  these  effects.    The  questions  here  are 
whether  the  effects  increase  with  increases  in  systematic  error  and  how  large 
are  the  effects.    The  next  subsection  deals  with  the  first  level  of  analysis. 
3.1.1    Direction  of  Effects 

The  directional  effects  of  systematic  measurement  error  on  CL  and  PL  were 
initially  analyzed  for  243  cases,  each  case  being  one  combination  of  values 
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for  the  three  parameters:  performance  specification  limits  (kj=k2);  standard 
deviation  of  measurement  error  (ae);  and  test  tolerance  adjustment  (TA).  The 
values  used  for  these  three  parameters  are  given  in  table  3.1,  along  with  the 
four  systematic  error  values  used  to  analyze  each  case. 

Because  an  increase  in  systematic  measurement  error  represents  a 
deterioration  in  the  accuracy  performance  of  the  test  equipment,  one's 
intuitive  expectation  is  that  greater  probabilities  of  wrong  decisions,  CL  and 
PL,  would  result.     Indeed,  PL  behaved  as  expected  for  all  of  the  cases 
analyzed,  while  CL  did  so  for  most  cases.     The  only  exceptions  occurred  for 
large  negative  values  of  TA  (i.e.,  when  the  test  limits  lie  considerably 
outside  the  UUT  performance  specification  limits) ,  where  CL  decreased  as  a 
result  of  increasing  the  systematic  measurement  error.     To  indicate  the  bounds 
for  these  exceptions,  it  is  useful  to  examine  the  critical  values  of  the  ratio 
of  TA  divided  by  the  standard  deviation  of  measurement  error  (i.e.,  TA/a£). 
Critical  values  are  to  be  interpreted  as  those  values  of  the  ratio,  TA/ae , 
below  which  CL  began  to  decrease  with  increasing  systematic  error.  These 
critical  ratios  are  presented  in  table  3.2  for  the  cases  analyzed.    That  is, 
for  values  of  TA/oe  below  (i.e.,  larger  in  a  negative  direction  than)  the 
critical  value  given  in  the  table,  CL  decreased  when  the  systematic  error  was 
increased.    The  data  in  table  3.2  suggest  two  general  patterns.    First,  the 
critical  value  of  the  ratio  appears  to  increase  (i.e.,  assumes  a  smaller 
negative  value)  for  larger  performance  limits.     Secondly,  the  critical  ratio 
tends  to  decrease  in  value  (i.e.,  assume  a  larger  negative  value)  for  smaller 
random  errors. 
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Table  3.1    Parameter  Values  and  Systematic 

Error  Values  Used  to  Analyze  the  Direction  of 
the  Effect  of  Systematic  Error  on  CL  and  PL 


Characteristic  Symbol 


Values 


Performance 
Specification 

Standard  Deviation 
of  Measurement 
Error 


k1-k2  1,2,3 


1/2,1/3,1/4,1/5,1/6,1/7,1/8,1/9,1/10 


Tolerance 
Adjustment 


TA 


0,  ±0.5ae,  ±1.0oe,  ±1.5oe,  +2.0oe 


Systematic  ye 
Measurement  Error 


0,  0.5oe,  1.0oe,  1.5o{ 
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Table  3.2    Critical  Values  of  TA/oe  Below  Which  CL  Decreases 
with  Increasing  Systematic  Error,  for  Various 
Symmetric  Performance  Limits  and  Standard  Deviations 
of  Measurement  Error. 


Standard  Deviation  Performance  Limits 


of  Measurement 
Error 

krk2=l 

k!=k2=2 

k2=k2=3 

0.500 

-0.5 

-0.5 

0.0 

0.333 

-1.0 

-0.5 

-0.5 

0.250 

-1.0 

-0.5 

-0.5 

0.200 

-1.0 

-1.0 

-0.5 

0.167 

-1.0 

-1.0 

-1.0 

0.143 

-1.5 

-1.0 

-1.0 

0.125 

-1.5 

-1.0 

-1.0 

0.111 

-1.5 

-1.0 

-1.0 

0.100 

-1.5 

-1.0 

-1.0 
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Because  of  this  contrary  behavior  of  CL  for  large  negative  values  of  TA, 
an  improvement  in  accuracy  achieved  by  reducing  systematic  measurement  error 
could  theoretically  result  in  an  increase  in  the  combined  costs  of  CL  and  PL. 
For  the  range  of  TA  values  defined  by  table  3.2,  PL  will  definitely  decrease, 
but  CL  will  increase.  Thus,  the  net  economic  result  could  be  positive  or 
negative,  depending  on  the  relative  sizes  of  the  offsetting  changes  in  CL  and 
PL  and  on  the  relative  economic  costs  of  each  occurrence  of  CL  and  PL. 

Fortunately,  this  ambiguity  concerning  the  overall  economic  effects  of 
reducing  bias  can  be  resolved  for  all  the  measurement  cases  analyzed.     In  the 
first  place,  the  problem  does  not  arise  very  often  because  large  negative  TA 
values  are  not  commonly  used  in  practice.     This  is  because  in  most 
applications  the  cost  consequences  of  an  occurrence  of  PL  are  less  than  those 
of  CL.     For  example,  the  rejection  of  a  good  item  (i.e.,  PL)  may  merely  result 
in  the  costs  of  unnecessary  retesting  or  the  scrapping  of  a  good  component, 
while  the  acceptance  of  a  bad  item  (i.e.,  CL)  could  result  in  failure  in  the 
field  of  an  entire  system  with  the  associated  costs  of  field  diagnosis  and 
repair  or  shipment  to  and  repair  in  a  test  and  repair  depot.    The  greater  the 
relative  cost  of  CL  versus  PL,  the  more  compelling  is  the  need  to  guard 
against  occurrences  of  CL  by  employing  a  larger  (in  the  positive  direction) 
value  for  TA. 

A  more  definitive  resolution  of  this  ambiguity  can  be  reached  by 
appealing  to  an  explicit  relationship  between  the  value  of  TA  and  the  relative 
cost  of  CL  versus  PL.    Grubbs  and  Coon  derived  such  an  explicit  relationship 
by  assuming  that  the  objective  of  test  policy  is  to  minimize  the  expected 
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value  of  the  total  cost  of  measurement  error,!  which  is  defined  as  follows: 

TC  =  CCL  •  CL  +  CPL  •  PL     ,  (3.1) 

where 

TC  =  the  expected  value  of  the  total  cost  of  measurement  error  for  each 
unit  tested; 

^CL  53  tne  economic  cost  of  one  occurrence  of  CL; 

CpL  -  the  economic  cost  of  one  occurrence  of  PL;  and 

CL,PL  are  as  previously  defined. 
Since  CL  and  PL  are  both  functions  of  TA,  Grubbs  and  Coon  partially 
differentiated  this  expression  for  TC  with  respect  to  TA.    This  procedure  led 
to  the  following  equation  that  determines  the  value  of  TA  which  minimizes  TC: 

G(Zl)  +  G(z2)  -  CPL/(CCL  +  CPL),  (3.2) 

where 

zi  =  -(1-P2)0-5  (k  +  auTA/ae2); 

z2  =  -(l-p2)°-5[k+2k(au/ae)2-0uTA/oe2]; 


k  =  kj  ■  k2,  standardized  performance 

in  expression  (2.5)  above;  and 
G(z)  =  standardized  univariate  normal 
as  defined  in  expression  (2.4) 


specification  limits  as  defined 

cumulative  distribution  function, 
above. 


Frank  E.  Grubbs  and  Helen  J.  Coon,  "On  Setting  Test  Limits  Relative  to 
Specification  Limits,"  p.  20. 
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One  application  of  this  equation  is  to  select  the  cost-minimizing  value 
of  TA  for  a  given  set  of  values  for  all  the  parameters:     CCL,  CpL,  ou,  oe ,  and 
k.    Another  application,  one  more  useful  in  resolving  the  ambiguity  concerning 
the  economic  effects  of  reducing  bias,  is  to  solve  for  that  value  of  the 
relative  cost  ratio,  Cq^/Cpl,  which  satisfies  the  equation  when  the  parameter 
values  listed  in  table  3.1  are  used  for  k,  o£ ,  ou,  and  TA.*    The  resulting 
cost  ratio  gives  the  relative  cost  consequences  of  a  unit  of  CL  and  PL  that 
would  have  to  prevail  in  order  for  the  TA  value  used  in  each  case  studied  to 
be  optimally  selected.     In  other  words,  if  it  is  assumed  that  the  TA  values 
for  each  case  are  selected  with  a  view  to  minimizing  TC,  then  the  equation 
permits  us  to  specify  the  underlying  relative  cost  ratio,  Cci/CpL* 

Following  this  procedure,  one  evaluates  the  two  G  functions  for  the 
values  of  zj  and  Z2  specified  by  the  parameter  values  for  k,  ae ,  au,  and  TA, 
and  then  sums  them.    This  sum  equals  the  righthand  side  of  the  equation, 
Cpl/(Ccl+Cpl) '     By  rearranging  terms,  one  obtains  the  following  expression  for 
the  relative  cost  of  CL  versus  PL: 

CCL/CPL  "  [G<zi)  +  G(z2)]-1  -  1.  (3.3) 

Investigating  the  direction  of  the  effect  that  reducing  bias  has  on  total 

measurement  error  costs  is  properly  accomplished  by  determining  the  sign  of 

the  partial  derivative  of  costs  with  respect  to  bias.    Differentiating  TC  as 

given  in  expression  (3.1),  one  obtains: 

9TC    =    CCL  •  _3CL  +  CPL  •  9PL     .  (3.4) 
3ye  3ye  3ue 


Though  not  explicitly  given  in  table  3.1,  <JU=1  for  all  the  cases  tested. 
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The  natural  expectation  is  that  a  reduction  in  the  bias  would  lead  to  lower 
total  measurement  error  costs.     This  expectation  is  fulfilled  provided  the 
partial  derivative  expression  is  positive.     Except  for  the  case  of  large 
negative  TA,  when  3CL/3ue  becomes  negative,  both  terms  of  the  expression  are 
positive,  so  that  the  derivative  is  positive  and  expectations  are  fulfilled. 
For  those  cases  in  which  3CL/3u£  is  negative,  the  derivative  will  be 
positive  if  and  only  if  the  absolute  value  of  first  product,  Cq^  •  3CL/3y£  ,  i 
smaller  than  the  second  product,  Cp^  •  3PL/3y£.     This  requirement  is 
equivalent  to  the  condition: 


ccl/cpl 


3PL/3ye 
3CL/3ue 


(3.5) 


An  analysis  was  conducted  for  all  those  cases  for  which  3CL/3y£  turned 
out  to  be  negative.     In  every  case,  the  condition  (3.5)  was  fulfilled,  and  by 
a  substantial  margin.     To  illustrate  these  results,  the  following  three 
figures  present  graphical  comparisons  of  the  two  ratios  of  expression  (3.5) 
for  the  case  of  TA/o£  =  -2.0. 1     For  the  figures  3.1,  3.2,  and  3.3,  the  values 
of  the  symmetric  performance  specification  limits  are  set  at  one  (k=l),  two 
(k=2),  and  three  (k=3)  standard  deviations,  respectively,  above  and  below  yu. 
The  results  are  similar  for  the  other  three  negative  values  used  for  TA/o£ 
(i.e.,  -1.5,  -1.0,  and  -0.5).     In  each  figure,  the  solid  line  indicates  the 


These  figures  were  derived  using  DATAPLOT,  the  graphical  analysis  computer 
software  package  developed  by  James  J.  Filliben  of  the  National  Bureau  of 
Standards.     For  a  summary  description  of  this  powerful  package,  see 
J.  J.  Filliben,  "DATAPLOT  —  An  Interactive  High  Level  Language  for  Graphics 
Non-Linear  Fitting,  Data  Analysis,  and  Mathematics,"  Computer  Graphics , 
Vol.  15,  No.  3  (August  1981),  pp  199-213. 
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value  of  the  relative  cost  ratio,  Cci/Cpi,,  as  calculated  from  equation  (3.3). 
The  data  points  (designated  by  dots)  are  absolute  values  of  the  relative 
change  ratio,  A PL/ ACL,  where 

APL  =  (PL    »0.5a  >    ~  <PLu  =0>   »  and 
e         e  e 

ACL  =  (CLU  =o.5o  )  ~  (CLu  =o) • 
e         e  e 

The  horizontal  axis  represents  the  relative  measurement  precision  of  each  case 
analyzed  as  indicated  by  the  ratio,  ou/oe.    As  each  figure  demonstrates,  the 
relative  cost  ratio,  C^/CpL,  is  consistently  less  than  the  absolute  value  of 
the  change  in  PL  relative  to  CL  for  the  entire  range  of  measurement  precision 
analyzed.     Thus,  for  these  cases  the  condition  as  stated  in  expression  (3.5) 
is  fulfilled,  which  demonstrates  that  an  increase  in  ye  leads  to  greater 
measurement  error  costs,  and  conversely,  that  a  decrease  in  \ie  leads  to  lower 
error  costs. 

3.1.2    Pattern  and  Magnitude  of  Effects 

Besides  the  direction  of  the  effects  that  changing  measurement  bias  has 
on  CL  and  PL,  the  general  pattern  and  order  of  magnitude  of  these  effects  need 
to  be  analyzed.     For  this  purpose,  the  computer  program  BINORMAL.CL-PL  was  run 
to  test  additional  cases  which  permit  smaller  increments  in  the  values  for 
systematic  measurement  error.     There  are  60  of  these  cases,  each  case  being 
one  combination  of  the  three  parameters:     performance  specification  limits  (kj 
*  ^2^5  standard  deviation  of  measurement  error  (oe);  and  test  tolerance 
adjustment  (TA) .     The  values  used  for  these  parameters  are  presented  in  table 
3.3,  along  with  the  set  of  nine  systematic  measurement  error  (ue )  values  used 
to  analyze  each  case.    As  was  true  for  the  computer  runs  listed  in  table  3.1, 
the  value  of  ou  for  all  these  cases  is  also  unity.     Partial  results  of  these 
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Table  3.3    Parameter  Values  and  Systematic  Error 
Values  Used  to  Analyze  the  Pattern  and 
Magnitude  of  the  Effect  of  Systematic 
Error  on  CL  and  PL. 


Characteristic  Symbol 

Performance  ^1=^2 
Specification 

Standard  Deviation  o£ 
of  Measurement 
Error 

Tolerance  TA 
Adjustment 


Values 


1,  2,  3 


0.1,  0.25,  0.5,  1.0 


0,  0.5oe,  1.0oe,  1.5ae,  2.0oe 


Systematic 
Measurement 
Error 


0,  0.25oe,  0.5oe,  0.75oe,  1.0c€, 
1.25oe,  1.5oe,  1.75oe,  2.0oe 
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runs  are  presented  in  the  figures  B.l  through  B.6  of  appendix  B  for  the  cases 
in  which  the  performance  specification  limits  equal  two  standard  deviations  of 
u  (i.e.,  kj=lc2=2).     The  pattern  and  magnitude  of  the  measurement  bias  effects 
on  CL  and  PL  can  be  illustrated  here  with  one  set  of  these  figures. 

Figure  3.4  presents  plots  of  the  value  of  CL  as  a  function  of  systematic 
measurement  error  for  the  case  of  ae=0.25au.     In  order  to  examine  the  effect 
of  TA,  five  plots  are  given  in  the  figure,  one  for  each  of  the  five  values  of 
TA  as  specified  in  table  3.3.    As  noted  in  section  3.1.1,  restricting  TA  to 
nonnegative  values  assures  that  there  is  a  positively  sloped  relationship 
between  systematic  measurement  error  and  CL.     Figure  3.4  confirms  this  rela- 
tionship, since  all  the  plots  indicate  that  CL  increases  as  systematic  error 
increases. 

Figure  3.5  presents  five  plots  of  PL  as  a  function  of  systematic  error 
for  the  case  of  0e=O.25au,  each  plot  corresponding  to  a  different  positive 
value  of  TA,  as  specified  in  table  3.3.    The  consistently  positive  relation- 
ship between  PL  and  systematic  error  that  was  noted  in  section  3.1.1  is  con- 
firmed in  all  these  plots  of  figure  3.5:     larger  systematic  measurement  errors 
result  in  greater  values  of  PL. 

Throughout  the  entire  range  of  values  for  systematic  error,  all  of  the  PL 
plots  and  most  of  the  CL  plots  (i.e.,  those  for  TA  >  1.0ae)  appear  to  be 
concave  upward  (i.e.,  positive  second  derivative  of  CL  and  PL  with  respect  to 
systematic  error).    This  upward  concavity  means  that  the  impacts  of  increases 
in  systematic  error  on  CL  and  PL  tend  to  increase  with  greater  errors.  For 
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0.0O"e    0.25CTe    0.50ae    0.75O"e    1.00(7e    1.25CTe    1.50ae    1.75ae  2.00ae 

SYSTEMATIC  MEASUREMENT  ERROR  (BIAS),  \xt 

Figure  3.4.     Consumer's  Loss  as  a  Function  of  Systematic  Measurement  Error 
(Bias),  for  Characteristic  Values  of  Test  Tolerance 
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O.OCTe    0.25<7e    0.50ae   0.75O"e  1.00(7e    1.25(7e    1.50O~e    1.75ae  2.00ae 
SYSTEMATIC  MEASUREMENT  ERROR  (BIAS),  /uf 

Figure  3.5.     Producer's  Loss  as  a  Function  of  Systematic  Measurement  Error 
(Bias),  for  Characteristic  Values  of  Test  Tolerance 
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example,  the  computed  values  on  which  figure  3.4  is  based  indicate  that,  for  a 
TA  value  of  1.0oe,  CL  increases  from  0.001961  to  0.002403  to  0.003125  when 
systematic  error  increases  from  0.25ae  to  0.5oe  to  0.75oe,  respectively. 
Thus,  the  first  increment  is  only  0.000442  (i.e.,  0.002403-0.001961),  while 
the  second  increment  is  nearly  twice  as  much,  0.000722  (i.e.,  0.003125- 
0.002403). 

In  spite  of  this  general  pattern  of  upward  concavity  for  the  plots  as 
shown,  one  can  observe  virtual  linearity  and  even  inflection  points  (reversal 
to  downward  concavity)  in  two  of  the  CL  plots  in  figure  3.4  (i.e.,  those  whose 
TA  values  are  0.5ae  and  0).     Indeed,  even  for  the  other  data  plots  inflection 
points  are  eventually  reached,  but  outside  the  range  of  systematic  error 
values  shown  in  the  figures. 

The  reason  for  the  eventual  decrease  in  slope  is  that  for  any  fixed  value 
of  the  performance  specification  (kj=k2),  a  certain  percentage  of  all  the  UUTs 
will  be  good  (within  specifications)  and  the  remaining  percentage  will  be  bad 
(outside  specifications).    These  percentages  represent  theoretical  limits  on 
PL  and  CL,  respectively.    The  actual  limits  on  CL  and  PL  for  any  particular 
test  situation  are  often  considerably  lower  than  these  theoretical 
percentages.!    The  plots  of  CL  and  PL  approach  these  actual  limits 
asymptotically.    The  decrease  in  slope  is  not  apparent  in  the  figures  for  PL 
because  the  systematic  measurement  errors  used  in  these  computations  are  not 


For  example,  when  jj£  is  positive  and  very  large,  the  test  will  accept  almost 
all  of  the  bad  UUTs  that  are  in  the  lower  rejection  region,  and  thus 
contribute  to  CL.    All  of  the  bad  UUTs  in  the  upper  rejection  region, 
however,  appear  much  worse  than  they  are  and  so  will  be  correctly  rejected 
and  will  not  contribute  to  CL.    Therefore,  in  this  case  the  actual  limit  on 
CL  is  only  half  of  the  theoretical  limit  (i.e. ,  half  of  the  total  percentage 
of  UUTs  that  are  bad). 
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large  enough  for  the  plots  to  reach  their  points  of  inflection.     For  some  of 
the  CL  plots  the  points  of  inflection  can  be  observed  because  they  tend  to 
occur  at  lower  error  values  for  CL  than  for  PL. 

Comparison  of  figures  3.4  and  3.5  reveals  that  for  a  given  size 
systematic  error,  the  value  of  TA  has  a  significant  but  opposite  effect  on  CL 
and  PL.     For  example,  a  decrease  in  TA  increases  CL  but  decreases  PL.  The 
relationship  between  CL  and  TA  makes  sense  if  one  remembers  that  TA  affects 
how  strict  the  test  is;  the  smaller  TA  is,  the  wider  (i.e.,  more  accepting) 
the  test  specifications  become.     The  reason  for  the  increase  in  CL  is  as 
follows.    A  decrease  in  TA  leaves  the  number  of  bad  products  unchanged,  but 
increases  the  likelihood  of  a  product's  being  accepted.     Thus,  more  bad 
products  will  be  accepted,  so  CL  increases. 

The  converse  is  true  for  the  relationship  between  PL  and  TA.    A  decrease 
in  TA  decreases  the  likelihood  of  a  product's  being  rejected  by  widening  the 
test  specifications,  and  yet  has  no  effect  on  the  actual  number  of  good  units 
available  to  be  rejected.    As  a  result,  fewer  good  products  are  rejected,  so 
that  PL  decreases.     In  effect,  the  selection  of  a  value  for  TA  constitutes  a 
choice  in  the  trade-off  between  CL  and  PL.     Indeed,  the  optimal  selection  of 
TA  to  minimize  the  total  economic  consequences  of  CL  and  PL  is  the  basis  of 
equation  (3.2),  developed  above  in  section  3.1.1. 
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3.2.     Effects  of  Random  Measurement  Error 

The  analysis  of  the  effects  of  random  measurement  error  on  CL  and  PL 
parallels  the  analysis  of  systematic  error  followed  in  subsection  3.1.  The 
first  level  of  analysis  concerns  the  direction  of  the  effects,  while  the 
second  level  focuses  on  their  general  pattern  and  magnitude. 
3.2.1.    Direction  of  Effects 

The  directional  effects  of  random  measurement  error  on  CL  and  PL  were 
analyzed  for  108  cases,  each  case  being  one  combination  of  values  for  the 
three  parameters:     performance  specification  limits  (kj=k2);  systematic 
measurement  error  (ue);  and  test  tolerance  adjustment  (TA).    The  values  used 
for  these  parameters  are  shown  in  table  3.4,  along  with  the  nine  values  of  the 
standard  deviation  of  measurement  error  (oe)  used  to  analyze  each  case. 
Because  o£  is  a  variable  in  each  case  of  this  analysis,  the  values  of  TA  and 
ue  are  expressed  in  absolute  terms  rather  than  in  units  of  ae  as  was  done  when 
the  effects  of  systematic  error  were  analyzed.     For  all  these  cases,  the  value 
of  au  is  assumed  to  be  unity. 

One  would  expect  an  increase  in  ae  to  lead  to  greater  probabilities  of 
incorrect  decisions,  in  terms  of  both  greater  CL  and  greater  PL.     Indeed,  for 
those  cases  most  likely  to  occur  in  test  situations,  CL  increased  as  oe 
increased.     This  expectation  is  fulfilled  as  long  as  the  systematic  error  is 
less  than  or  equal  to  the  test  tolerance  adjustment.     This  finding  can  be 
stated  in  the  form  of  a  sufficient 1  condition.     That  is, 


There  are  a  few  cases  in  which  ue  >  TA  and  still  3CL  >  0. 

3ae 
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Table  3.4    Parameter  Values  and  Standard  Deviation  of 

Measurement  Error  Values  Used  to  Analyze  the 
Direction  of  the  Effect  of  the  Standard 
Deviation  of  Measurement  Error  on  CL  and  PL. 


Characteristic  Symbol 

Performance  ^1=^2 
Specification 

Systematic  ue 
Measurement  Error 

Tolerance  TA 
Adjustment 


 Values  

1,  2,  3 

0,  0.5,  1.0,  1.5 

0,  +0.25,  +0.5,  +0.75,  +1.0 


Standard  oe  1/2,  1/3,  1/4,  1/5,  1/6,  1/7,  1/8, 

Deviation  of  1/9,  1/10 

Measurement 
Error 
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3CL  >  o      if  He  <  TA.1  (3.8) 
3ae 

In  those  actual  test  situations  in  which  the  order  of  magnitude  of  ue  is 
known,  it  is  likely  that  TA  will  be  set  larger  than  ue  in  order  to  control  the 
size  of  CL,  provided  the  economic  cost  of  each  CL  occurrence  is  significant. 
If,  however,  the  cost  of  CL  is  relatively  low  compared  with  that  of  PL,  then 
the  condition  given  in  expression  (3.8)  is  not  likely  to  be  fulfilled.  That 
is,  values  smaller  than  ue  or  even  negative  values  will  likely  be  assigned  to 
TA.     For  such  cases,  the  total  cost  of  measurement  error  as  specified  in 
equation  (3.1)  should  be  analyzed  along  the  lines  followed  in  subsection 
3.1.1.2 

An  increase  in  oe  is  also  expected  to  cause  an  increase  in  PL.  Once 

again,  for  the  most  likely  test  situations  these  expectations  are  fulfilled. 

For  all  the  cases  analyzed,  PL  consistently  increased  as  a  result  of 

increasing  o£  as  long  as  the  performance  specification  exceeded  the  sum  of  the 

tolerance  adjustment  and  the  systematic  measurement  error.    This  finding  can 

be  summarized  in  terms  of  the  condition  statement: 

3 PL  >  o      if  k*ou  >  TA+pe ,  (3.9) 
3oe 

where  k  is  the  number  of  standard  deviations  that  the  performance  limits 
lie  above  and  below  the  mean  for  a  symmetric  test.    That  is, 
k-ou  =  sU_^u  =  Hu-Sl  =  0.5(Su-SL). 


*The  partial  derivative  could  theoretically  be  zero  for  the  unlikely  case  in 
which  the  tolerance  adjustment  equals  the  performance  specification.  In 
this  case,  CL  remains  zero  because  there  is  no  acceptance  region  and  the 
only  error  possible  is  PL,  which  remains  at  its  maximum  value. 

2This  analysis  is  proposed  as  the  subject  of  future  research. 


39 


The  condition  stated  in  expression  (3.9)  will  almost  always  be  fulfilled 
in  realistic  test  situations  because  the  value  k*ou  represents  one  half  of  the 
entire  range  enclosed  within  the  upper  and  lower  performance  specification 
limits.     The  sizes  of  TA  and  ye  derive  from  the  distribution  of  the  measure- 
ment error,  e,  which  typically  has  a  narrower  range,  by  an  order  of  magnitude, 
than  that  of  the  UUT  performance  characteristic,  u.     Consequently,  even  the 
sum,  TA  +  ye ,  will  be  considerably  smaller  than  half  the  performance  range 
under  normal  circumstances.     It  is  interesting  to  note,  however,  that  it  is 
theoretically  possible  for  an  increase  in  oe  caused  by  a  decrease  in  precision 
to  lead  to  a  decrease  in  the  probability  of  rejecting  good  units  (PL).  An 
important  objective  for  future  research  is  to  establish  more  exactly  the 
conditions  for  assuring  that  PL  increases  as  a£  increases,  either  through  more 
detailed  simulations  or  the  application  of  calculus. 

3.2.2.    Pattern  and  Magnitude  of  Effects 

The  pattern  and  magnitude  of  the  effects  on  CL  and  PL  of  changes  in 
measurement  precision  is  established  by  running  the  computer  program 
BINORMAL.CL-PL  for  a  wide  range  of  cases  in  which  a£  is  assigned  steadily 
increasing  values.    Forty-five  such  cases  were  analyzed,  each  case  being  one 
combination  of  the  three  parameters:     symmetric  performance  specification 
limits  (kj=k2);  systematic  measurement  error  (ye);  and  test  tolerance  adjust- 
ment (TA).    The  values  used  for  these  parameters  ace  presented  in  table  3.5, 
along  with  the  set  of  nine  oe  values  used  to  analyze  each  case.    As  for  all  of 
the  cases  analyzed  in  this  section,  the  value  of  au  is  assumed  to  be  unity. 
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Partial  results  of  this  analysis  are  shown  in  the  plots  of  figures  B.7  through 
B.12  of  appendix  B  for  those  cases  in  which  the  performance  specification 
limits  equal  two  standard  deviations  of  u  (i.e.,  kj=k2=2).     Both  the  general 
pattern  and  the  size  of  the  effects  of  ae  on  CL  and  PL  are  illustrated 
here  with  one  set  of  these  plots. 


Table  3.5    Parameter  Values  and  Standard  Deviation  of 

Measurement  Error  Values  Used  to  Analyze  the 
Pattern  and  Magnitude  of  the  Effect  of  the 
Standard  Deviation  of  Measurement  Error  on 
CL  and  PL. 


Characteristic 

Performance 
Specification 


Symbol 
ki=k2 


Values 


1,  2,  3 


Systematic  ue 
Measurement  Error 


0,  0.25,  0.50 


Tolerance 
Adjustment 


TA 


0,  0.25,  0.50,  0.75,  1.0 


Standard  ae  0.1,  0.2,  0.3,  0.4,  0.5,  0.6,  0.7, 

Deviation  of  0.8,  0.9 

Measurement 
Error 
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Figure  3.6  presents  plots  of  the  value  of  CL  as  a  function  of  the 
standard  deviation  of  measurement  error  for  the  case  of  systematic  error  equal 
to  0.25.     To  analyze  the  effect  that  TA  has  on  CL,  four  separate  plots  are 
given,  based  on  the  values  specified  for  TA  in  table  3.5.1    F0r  an  Qf  these 
plots  in  figure  3.6  the  condition  in  expression  (3.8)  is  fulfilled,  so  that  CL 
is  a  positively  sloped  function  of  oe . 

The  curvature  of  all  the  plots  in  figure  3.6  varies  depending  on  the 
value  of  TA  and  the  range  of  interest  for  ae .    For  high  values  of  TA  (i.e.,  ^> 
0.5)  the  plots  are  somewhat  concave  upward  initially  and  then  reach  inflection 
points  and  become  concave  downward  as  a£  increases.     The  points  of  inflection 
occur  at  lower  values  of  ae  ,  the  lower  the  value  of  TA  used.     Indeed,  for  the 
lowest  value  of  TA  (i.e.,  0.25),  the  inflection  point  appears  to  occur  for 
values  of  oe  less  than  0.1.     The  significance  of  upward  concavity  is  that  the 
impacts  of  increases  in  oe  on  CL  tend  to  increase  with  greater  oe .  For 
example,  the  computed  values  underlying  the  plot  for  a  TA  value  of  1.0 
indicate  that  CL  increases  from  0.000200  to  0.000595  to  0.001200  when  o£ 
increases  from  0.4  to  0.5  to  0.6,  respectively.     Thus,  the  first  increment  is 
only  0.000395  (i.e.,  0.000595  -0.000200),  while  the  second  increment  is 
0.000605. 

In  contrast  to  this  observed  upward  curvature,  for  a£  values  in  excess  of 
0.6,  a  general  pattern  of  approximate  linearity  is  observed  for  all  of  the 
plots.     In  this  range,  therefore,  the  impact  on  CL  of  increasing  ae  is 
relatively  constant. 

*0nly  four  of  the  five  values  of  TA  given  in  table  3.5  are  used  in  these  plots 
because  for  TA=0  the  condition  given  in  expression  (3.8)  is  violated  (i.e., 
Me»0.25>TA=0) .     The  plot  of  CL  for  this  value  of  TA  would  have  a  negative 
slope. 
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Figure  3.7  presents  the  four  plots  of  PL  as  a  function  of  the  standard 
deviation  of  measurement  error  for  the  same  values  of  TA  as  used  in  figure  3.6 
for  CL.     Because  the  condition  specified  in  expression  (3.9)  is  fulfilled  for 
all  the  cases  represented  by  these  plots,  they  are  positively  sloped  through- 
out their  range. 

The  curvature  of  all  the  plots  in  figure  3.7  is  such  that  they  are 
concave  upward  over  the  entire  range  of  a£  values  analyzed.     Since  it  is 
unlikely  that  a£  would  exceed  unity  (the  value  of  au) ,  this  concavity  can  be 
said  to  prevail  for  most  realistic  test  situations. *     As  is  true  for  the 
effects  on  CL  discussed  above,  the  upward  concavity  means  that  the  impacts  of 
increases  in  o£  on  CL  tend  to  increase  with  greater  ae .    For  example,  the 
computed  values  underlying  figure  3.7  indicate  that  for  a  TA  value  of  0.25,  CL 
increases  from  0.047473  to  0.053500  to  0.062408  when  a£  increases  from  0.1  to 
0.2  to  0.3,  respectively.    Thus,  the  first  increment  is  only  0.006027,  while 
the  next  increment  is  0.008908. 


The  curvature  of  these  plots  where  oe<0.1ou  was  not  analyzed  here. 
Extrapolation  of  these  plots  back  toward  the  origin  would  suggest  that  the 
curvature  decreases  with  decreasing  slope. 
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4.  Summary  and  Application 

In  this  report  an  economic  model  has  been  presented  which  traces  the 
effects  of  alternative  levels  of  accuracy  in  testing  and  measurement  on  the 
correctness  of  acceptance  and  rejection  decisions.     The  model  expresses  two 
probabilities,  that  of  accepting  a  bad  unit  (CL) ,  and  that  of  rejecting  a  good 
unit  (PL),  as  explicit  functions  of  systematic  measurement  error  (ye )  and  the 
standard  deviation  of  random  measurement  error  (o£).     These  functions  can  be 
used  to  calculate  directly  the  benefits,  as  measured  by  lower  probabilities  of 
errors,  from  reduced  systematic  and/or  random  errors.    When  applied  to  a  case 
study,  an  economic  value,  in  terms  of  dollars  saved  per  time  period,  can  be 
established  for  these  reduced  probabilities.     The  economic  value  depends  on 
the  number  of  tests  conducted  in  a  time  period  and  on  the  particular  test 
situation  being  analyzed.    Multiplication  of  each  probability  reduction  by  the 
number  of  items  to  be  tested  gives  the  expected  reduction  in  the  number  of 
occurrences  of  each  type  of  test  error.    The  economic  value  of  each  reduced 
error  occurrence  can  then  be  determined  by  identifying  and  categorizing  the 
cost  consequences  of  single  CL  and  PL  occurrences.     Separate  estimates  of  the 
value  of  each  cost  consequence  have  to  be  derived.     Examples  of  such  cost  con- 
sequences for  a  CL  occurrence  are  field  repair  costs,  accidents  due  to  equip- 
ment failure,  and  possible  effects  on  readiness  and  mission  success  rates  in 
military  applications.     Examples  of  cost  implications  for  a  PL  occurrence  are 
unnecessary  diagnostic  and  repair  costs,  the  costs  of  unnecessary  retesting 
(including  packing  and  shipping  to  higher  echelon  laboratories),  excessive 
inventory  of  parts  and  UUTs  because  of  downtime,  and  reduced  production 
yields. 
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The  applicability  as  well  as  the  dollar  value  of  each  of  these  and  the 
many  other  cost  implications  resulting  from  occurrences  of  CL  and  PL  can  vary 
greatly  depending  on  the  particular  test  situation  being  analyzed.  Examples 
of  the  many  test  applications  possible  are  production  testing  both  at 
intermediate  stages  and  at  the  final  product  stage  (i.e.,  outgoing  tests), 
incoming  testing  of  components  to  be  used  in  manufacturing,  and  maintenance 
testing.     The  economic  model  presented  in  this  report  can  be  used  to  analyze 
all  of  these  test  applications,  provided  the  basic  nature  of  the  measurement 
process  is  analog  rather  than  digital.     That  is,  the  principal  characteristic 
of  the  UUT  that  is  being  measured  must  be  a  continuous  variable,  rather  than 
a  binomial  or  other  type  of  discrete  variable.     This  requirement  is  not  too 
restrictive,  however,  because  even  when  semiconductor  devices  that  are 
primarily  digital  in  design  are  being  tested,  some  critical  analog 
measurements  must  be  made. 

This  economic  model  can  be  applied  to  a  wide  variety  of  fields,  including 
metrology,  research  planning,  maintenance  and  procurement  policy,  and  product 
quality  assurance  functions.     To  illustrate  metrology  research  applications, 
the  magnitudes  of  the  reductions  in  CL  and  PL  that  would  result  from 
alternative  calibration  support  strategies  can  be  computed  by  the  model  and 
then  used  to  assist  program  managers  in  selecting  test  methods  and  calibration 
support  systems  that  have  the  highest  potential  payoff.     In  the  maintenance 
area,  the  model  can  be  used  to  optimize  the  frequency  and  quality  of 
calibration  procedures  with  existing  test  equipment.     In  the  area  of 
procurement  policy,  the  model  can  be  used  to  select  the  best  new  test 
equipment  through  a  comparison  of  additional  costs  with  the  economic  benefits 
of  expected  accuracy  improvements.     In  quality  assurance,  the  model  can  be 
applied  to  determine,  for  a  particular  product  and  test  environment,  which 
test  specifications  optimize  the  economic  tradeoff  between  CL  and  PL. 
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Appendix  A.  Binormal.CL-PL,  an  Interactive  Computer  Program  for  Calculating 
Consumer's  Loss  and  Producer's  Loss  in  Testing 


A.l  Purpose 

BINORMAL.CL-PL  is  an  interactive  FORTRAN  program  developed  for  use  in 
measuring  the  benefits  of  improved  accuracy  of  test  equipment.!    Two  types  of 
accuracy  improvements  can  be  analyzed:     (1)  a  reduction  of  the  systematic 
measurement  error  of  test  equipment  (i.e.,  a  reduction  of  the  bias  present  in 
the  measurement  results);  and  (2)  a  reduction  of  the  standard  deviation  of  the 
measurement  error  of  the  test  equipment  (i.e.,  an  increase  in  measurement 
precision).    These  improvements  might  be  achieved,  for  example,  through  better 
test  operating  procedures,  more  controlled  environmental  conditions  during 
testing,  more  frequent  calibration  procedures,  as  well  as  through  the 
acquisition  of  more  accurate  test  equipment.    When  provided  with  parameters 
that  fully  characterize  the  distribution  of  the  UUT  performance2  and  the 
accuracy  of  the  test  equipment,  BINORMAL.CL-PL  calculates  the  probability  of 
accepting  a  UUT  that  does  not  conform  to  its  performance  specifications  (i.e., 
Consumer's  Loss  or  CL)  and  the  probability  of  rejecting  a  UUT  that  does 
conform  to  its  performance  specifications  (i.e.,  Producer's  Loss  or  PL).  The 
option  is  available  for  conducting  a  sensitivity  analysis  of  the  results  by 


The  software  documentation  in  this  appendix  follows  the  principal  Federal 
Information  Processing  Standards  (FIPS)  found  in  Guidelines  for  Documentation 
of  Computer  Programs  and  Automated  Data  Systems,  FIPS  Publication  #38,  U.S. 
Department  of  Commerce,  National  Bureau  of  Standards,  February  15,  1976. 

UUT  performance  refers  to  the  value  of  the  particular  UUT  attribute  that  is 
being  measured  by  the  test  equipment. 
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varying  the  parameters  relating  to  the  accuracy  of  the  test  equipment  and  the 
setting  of  the  test  specifications. 

BINORMALoCL-PL  can  be  helpful  in  planning  research,  making  maintenance 
and  procurement  decisions,  and  solving  certain  quality  control  problems.  The 
results  of  the  program  can  be  used  to  guide  decisions  regarding  which  test 
methods  and  calibration  support  strategies  would  have  the  highest  payoff.  The 
program  also  could  be  used  to  optimize  the  frequency  and  quality  of 
calibration  procedures  with  existing  test  equipment,  as  well  as  to  select  new 
test  equipment  by  comparing  additional  costs  with  expected  accuracy 
improvements.    Finally,  in  the  area  of  quality  control,  the  sensitivity 
analysis  capability  of  the  program  can  be  used  to  establish  for  a  particular 
product  the  test  specifications  that  optimize  the  trade-off  between  CL  and 
PL. 

A. 2  Scope 

BINORMAL . CL-PL  can  be  run  on  any  computer  with  a  FORTRAN  77  compiler 1  and 
access  to  routines  supplied  by  the  International  Mathematical  and  Statistical 
Libraries,  Inc.  (IMSL).2    The  program  is  quite  flexible  in  a  number  of  ways. 
In  the  first  place,  it  can  be  applied  to  the  three  most  commonly  encountered 
test  situations:     two-tail  tests;  upper-tail  tests;  and  lower-tail  tests. 3 
In  a  two-tail  test,  the  UUT  performance  is  expected  to  be  between  established 


1 American  National  Standards  Programming  Language,  FORTRAN,  ANSI  X3. 9-1 978. 

2 Documentation  for  the  two  routines  used  in  the  program,  MDNOR  and  MDBNOR,  can 
be  found  in  International  Mathematical  and  Statistical  Libraries,  Inc., 
The  IMSL  Library,  9th  Edition,  Vol.  3  (Houston,  TX,  1982). 

^The  only  test  situations  that  cannot  be  analyzed  by  the  program  are  those  in 
which  the  acceptable  values  are  in  any  way  separated  from  one  another  by 
unacceptable  values.     For  instance,  if  the  extreme  values  (i.e.,  the  two 
tails)  of  the  UUT  performance  distribution  constituted  acceptable  regions 
while  the  values  close  to  the  mean  constituted  the  unacceptable  region,  then 
this  program  could  not  be  used. 
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upper  and  lower  performance  specifications  in  order  to  be  acceptable.  An 
upper-tail  test  is  one  in  which  the  UUT  performance  is  expected  to  be  below  a 
given  upper  performance  specification  to  be  acceptable,  while  a  lower-tail 
test  is  one  in  which  the  UUT  performance  is  expected  to  be  above  a  lower 
performance  specification  to  be  acceptable.     The  program  is  also  flexible  in 
that  the  test  specifications,  that  is,  the  cut-off  values  for  the  measured  UUT 
performance,  can  be  set  anywhere  in  relation  to  the  performance 
specifications.    Moreover,  when  the  program  is  used  for  a  two-tail  test,  the 
performance  specifications  need  not  be  symmetric  around  the  mean  UUT 
performance.     There  is  also  great  flexibility  in  the  types  of  UUTs  and  test 
equipment  the  program  can  analyze.     The  only  requirement  is  that  the  UUT 
performance  and  the  test  equipment  measurement  error  both  must  be  assumed  to 
follow  normal  distributions,  which  implies  that  they  can  be  treated  as 
continuous  variables.    The  program  user  is  free  to  specify  any  means  and 
standard  deviations  for  these  normal  distributions. 1 
A. 3  Using  the  Program 

Once  the  user  has  entered  the  proper  control  statements  to  begin 
execution  of  the  program, 2  the  computer  prompts  the  user  for  each  input  value 
to  be  entered.    Table  A.l  lists  and  defines  the  symbols  used  for  the  input 
variables  in  this  appendix  and  in  the  program.     In  addition  to  entering  these 
values,  the  user  must  indicate  whether  the  test  has  two  tails,  an  upper  tail 
only,  or  a  lower  tail  only. 


•'■In  practice,  the  standard  deviation  of  the  measurement  error  will  typically 
be  less  than  that  of  the  UUT. 

^Since  control  statements  are  specific  to  the  computer  system  on  which  the 
program  is  run,  they  will  not  be  discussed  here. 
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The  output  of  the  program  is  a  listing  of  all  the  input  values  and  the 
corresponding  computed  probabilities,  CL  and  PL.    After  the  results  are 
printed  the  user  indicates  whether  or  not  a  sensitivity  analysis  is  desired. 
If  not,  the  program  ends.     If  sensitivity  analysis  is  requested,  the  user  is 
prompted  to  enter  new  values  for  ME,  SDE,  and  BU  and/or  BL  (depending  on  the 
test  situation).    New  values  for  CL  and  PL  are  calculated  and  the  new  results 
are  printed  in  the  same  format  as  before.    The  user  can  continue  entering  new 
values  as  many  times  as  desired.    A  sample  run  with  one  case  of  sensitivity 
analysis  is  shown  in  figure  A. 1. 

Table  A.l     Input  Variables  for  BINORMAL . CL-PL 


Symbol  Definition 


MU  Mean  of  the  UUT  performance  distribution. 

SDU  Standard  deviation  of  the  UUT  performance  distribution. 

ME  Mean  of  the  test  equipment  measurement  error  distri- 

bution, also  known  as  undetected  systematic  measurement 
error  or  bias. 

SDE  Standard  deviation  of  the  test  equipment  measurement 

error  distribution,  also  known  as  measurement  imprecision. 

SU  Upper  specification  limit  for  UUT  performance. 

SL  Lower  specification  limit  for  UUT  performance. 

BU  Upper  test  equipment  tolerance  factor. 1 

BL  Lower  test  equipment  tolerance  factor.* 


The  tolerance  factor  is  the  number  of  SDEs  between  the  UUT  performance 
specification  value  and  the  test  specification  value.     It  is  generally  used 
to  allow  for  measurement  error  of  the  test  equipment.    A  positive  factor 
means  that  the  test  specification  limits  are  more  stringent  (i.e.,  more 
likely  to  reject  UUTs)  than  the  unadjusted  performance  specification  limits. 
In  many  cases  the  tolerance  factor  is  zero,  which  means  that  the  test  speci- 
fication limits  are  set  equal  to  the  UUT  performance  specification  limits. 
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Figure  A.l.     Illustrative  Computer  Run  of  BINORMAL . CL-PL 


ENTER  MEAN  OF  MEASUREMENT  ERROR 
>.25 

ENTER  STANDARD  DEVIATION  OF  MEASUREMENT  ERROR 
>.5 

ENTER  MEAN  OF  TRUE  PERFORMANCE 
>0 

ENTER  STANDARD  DEVIATION  OF  TRUE  PERFORMANCE 
>1 

IS  THIS  A  TWO-TAIL  TEST  OF  PERFORMANCE?( Y/N) 
>Y 

ENTER  UPPER  PERFORMANCE  SPECIFICATION 
>2 

ENTER  LOWER  PERFORMANCE  SPECIFICATION 
>-2 

ENTER  UPPER  TOLERANCE  FACTOR  OF  ATE 
>1 

ENTER  LOWER  TOLERANCE  FACTOR  OF  ATE 
>1 

FOR: 

MEAN  OF  MEASUREMENT  ERROR  =  .25000000 

STANDARD  DEVIATION  OF  MEASUREMENT  ERROR  =  .50000000 
MEAN  OF  TRUE  PERFORMANCE  =  .00000000 

STANDARD  DEVIATION  OF  TRUE  PERFORMANCE  =  1.0000000 
UPPER  PERFORMANCE  SPECIFICATION  =  2.0000000 

LOWER  PERFORMANCE  SPECIFICATION  =  -2.0000000 

UPPER  TOLERANCE  FACTOR  OF  ATE  =  1.0000000 

LOWER  TOLERANCE  FACTOR  OF  ATE  =  1.0000000 

THEN  CONSUMER  LOSS  =  .003878 
AND  PRODUCER  LOSS  =  .148917 
WOULD  YOU  LIKE  TO  DO  A  SENSITIVITY  ANALYSIS?( Y/N) 
>Y 

ENTER  NEW  VALUES  FOR  MEAN  OF  MEASUREMENT  ERROR 

STANDARD  DEVIATION  OF  MEASUREMENT  ERROR  .UPPER  TOLERANCE  FACTOR  OF  ATE,  AND 
LOWER  TOLERANCE  FACTOR  OF  ATE  ,  ONE  VALUE  PER  PROMPT. 


>.125 
>.5 
>1 
>1 

FOR: 


MEAN  OF  MEASUREMENT  ERROR 

STANDARD  DEVIATION  OF  MEASUREMENT  ERROR 

MEAN  OF  TRUE  PERFORMANCE 

STANDARD  DEVIATION  OF  TRUE  PERFORMANCE 

UPPER  PERFORMANCE  SPECIFICATION 

LOWER  PERFORMANCE  SPECIFICATION 

UPPER  TOLERANCE  FACTOR  OF  ATE 

LOWER  TOLERANCE  FACTOR  OF  ATE 


.12500000 
.50000000 
.00000000 
1.0000000 
2.0000000 
-2.0000000 
1.0000000 
1.0000000 


THEN  CONSUMER  LOSS  =  .003214 
AND  PRODUCER  LOSS  =  .140143 
WOULD  YOU  LIKE  TO  DO  A  SENSITIVITY  ANALYSIS?( Y/N) 
>N 
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The  user  should  be  aware  that  there  is  no  capability  in  the  program  for 
evaluating  whether  the  numbers  entered  are  reasonable.     That  is,  there  is  no 
edit  checking  in  the  program.     Thus,  the  user  must  be  sure  that  the  values 
entered  represent  a  realistic  situation.     For  example,  if  the  values  entered 
produce  a  lower  test  specification  that  is  greater  than  the  upper  test 
specification,  then  the  program  will  not  fail  to  run,  but  may  result  in 
negative  values  for  CL  and/or  values  for  PL  that  are  greater  than  unity. 
A. 4    Program  Characteristics 

B I NORMAL. CL-PL  is  written  in  FORTRAN  77  with  external  routines  supplied 
by  IMSL.     These  routines,  MDNOR  and  MDBNOR,  give  single  precision  results. 
The  accuracy  of  MDNOR,  which  calculates  univariate  normal  probabilities,  is 
dependent  on  the  computer  system  being  used.     To  compute  the  CL  and  PL  values 
presented  as  illustrations  and  results  throughout  this  report,  the  authors 
employed  the  UNIVAC  1100/82  system  installed  at  NBS  in  Gaithersburg ,  Maryland. 
The  routine  MDNOR  would  be  accurate  to  the  eighth  digit  on  such  a  36  bit 
machine.1     Documentation  in  the  IMSL  Library  states  that  MDBNOR,  which 
calculates  bivariate  normal  probabilities,  gives  values  differing  by  less  than 
0.00001  from  the  values  in  Tables  of  the  Bivariate  Normal  Distribution 
Function  and  Related  Functions. 2  The  values  in  these  tables  have  a  maximum 
absolute  error  of  0.000002,3  so  that  the  total  absolute  error  for  each 
bivariate  normal  probability  computed  by  MDBNOR  is  less  than  0.000012.  Given 
these  accuracies  for  the  two  external  routines  used,  it  is  expected  that  the 
value  of  CL 

^Telephone  conversation  with  technical  staff  at  IMSL. 

^U.S.  Department  of  Commerce,  Tables  of  the  Bivariate  Normal  Distribution 
Function  and  Related  Functions,  National  Bureau  of  Standards,  Applied 
Mathematics  Series  50  (Washington,  D.C.,  1959). 

3Ibid. ,  p.  vi. 
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computed  by  BINORMAL . CL-PL  will  have  a  maximum  alsolute  error  of  0.000048, 
while  the  value  of  PL  would  have  a  maximum  absolute  error  of  0.00004804. 
The  following  formulas  are  used  to  calculate  CL  and  PL: 


RHO  =        SDU  ;      where  RHO  is  the  statistical  correlation 

between  the  UUT  performance  values  and  the 
/ SDU^+SDE7         observed  measurement  values; 

Kl  =  SU-MU; 
SDU 

K2  =  MU-SL; 
SDU 

Ql  =     SU-MU- (BU*SDE) -ME 

;  and 

/SDUZ+SDE* 
02  =  MU-SL-(BL*SDE)+ME 
/SDU*+SDE* 
For  a  two-tail  test, 


CL  -  L(-K1 ,Q2,RH0)-L(-K1 ,-Ql ,RHO)-L(-K2,-Q2,RHO)+L(-K2,01 ,RH0) , 

where    L(x,y,r)  is  the  joint  probability  that  X  <  x  and  Y  <  y  given  that  (X,Y) 
is  the  standardized  bivariate  normal  cumulative  distribution  function 
with  correlation  RHO  between  X  and  Y,  as  calculated  by  MDBN0R;  and 

PL  =  CL+G(K1)+G(K2)-G(Q1)-G(Q2) , 

where  G(x)  is  the  probability  that  X  <  x  given  that  X  is  the  standardized 
univariate  normal  cumulative  distribution  function,  as  calculated 
by  MDNOR. 

For  an  upper-tail  test, 


CL  =  G(-K1)-L(-K1 ,-01 ,RH0)  and 
PL  -  G(-Q1)-L(-K1 ,-Ql ,RH0). 


For  a  lower-tail  test, 


CL  =  G(-K2)-L(-K2,-Q2,RH0)  and 
PL  =  G(-Q2)-L(-K2,-Q2,RH0). 


A  listing  of  BINORMAL. CL-PL  is  shown  in  figure  A. 2. 
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Figure  A. 2.     Computer  Code  Listing  of  BI NORMAL. CL-PL 


ACID*BIN0RMAL(1).CL-PL(45) 

1  C  PROGRAM:    B I NORMAL. CL-PL 

2  C 

3  C  DATE  OF  LAST  REVISION:  12/22/83 

4  C 

5  C  DOCUMENTATION:    THIS  PROGRAM  CALCULATES  CONSUMER  LOSS  (CL)  AND 

6  C  PRODUCER  LOSS  (PL)  FOR  INPUTTED  LEVELS  OF  ACCURACY  OF  AUTOMATIC 

7  C  TEST  EQUIPMENT  (ATE).    THE  LOSSES  CAN  BE  CALCULATED  FOR  BOTH 

8  C  ONE-TAIL  AND  TWO-TAIL  TEST  SITUATIONS.    IT  IS  ASSUMED  THAT 

9  C  MEASURED  PERFORMANCE  FOLLOWS  A  BIVARIATE  NORMAL  DISTRIBUTION 

10  C  WHERE  MEASURED=TRUE+ERROR  IN  MEASUREMENT,  WHERE  TRUE  AND 

11  C  ERROR  ARE  NORMAL  DISTRIBUTIONS.    THE  VARIABLES  USED  ARE: 

12  C 

13  C  MEANE=ME=MEAN  OF  MEASUREMENT  ERROR 

14  C  SDE=STE=STANDARD  DEVIATION  OF  MEASUREMENT  ERROR 

15  C  MEANU=MU=MEAN  OF  TRUE  PERFORMANCE  OF  UNIT  UNDER  TEST 

16  C  SDU=STU=STAMDARD  DEVIATION  OF  TRUE  PERFORMANCE  OF  UUT 

17  C  SU=UPPER  PERFORMANCE  SPECIFICATION 

18  C  SL=LOWER  PERFORMANCE  SPECIFICATION 

19  C  BU=UPPER  ATE  TOLERANCE  FACTOR 

20  C  (BU*SDE=UPPER  TOLERANCE,  WHERE  BU>0  MEANS  UPPER  TEST  SPECIFICATION 

21  C  LIES  BELOW  THE  UPPER  PERFORMANCE  SPECIFICATION.) 

22  C  BL=LOWER  ATE  TOLERANCE  FACTOR 

23  C  ( BL*SDE=LOWER  TOLERANCE,  WHERE  BL>0  MEANS  LOWER  TEST  SPECIFICATION 

24  C  LIES  ABOVE  THE  LOWER  PERFORMANCE  SPECIFICATION.) 

25  C  ^STANDARDIZATION  OF  SU 

26  C  ^STANDARDIZATION  OF  SL 

27  C  Q1=STANDARDIZATI0N  OF  UPPER  TEST  SPECIFICATION 

28  C  Q2=STANDARDIZATI0N  OF  LOWER  TEST  SPECIFICATION 

29  C  RHO=CORRELATION  OF  MEASURED  PERFORMANCE  WITH  TRUE  PERFORMANCE 

30  C  P=BIVARIATE  NORMAL  PROBABILITY  OR  UNIVARIATE  NORMAL  PROBABILITY 

31  C  IER=ERROR  FLAG,  MEANING  RH0>=1.  IN  MDBNOR. 

32  C  STR1  TO  STR8  AND  SI  TO  S8  ARE  STRINGS  FOR  PRINTING. 

33  C  YESNO  AND  YN  ARE  USER'S  ANSWERS  TO  QUESTIONS. 

34  C 

35  C  ALGORITHMS  USED  ARE: 

36  C 

37  C  RHO=SDU/SQRT((SDU*SDU)+(SDE*SDE)) 

38  C  K1=(SU-MU)/SDU 

39  C  K2=(MU-SL)/SDU 

40  C  Q1=(SU-MU-(BU*SDE)-ME)/SQRT((SDU*SDU)+(SDE*SDE)) 

41  C  Q2=(MU-SL-(BL*SDE)+ME)/SQRT((SDU*SDU)+(SDE*SDE)) 

42  C  ALGORITHMS  FOR  CL  AND  PL  DEPEND  ON  WHAT  TYPE  OF  TEST  IS  USED. 

43  C 

44  REAL  MEANE , SDE ,MEANU , SOU 

45  CHARACTER*40  STR1,STR2,STR3,STR4,STR5,STR6,STR7,STR8 

46  CHARACTERS  YESNO 

47  DATA  STR5/' UPPER  PERFORMANCE  SPECIFICATION'/ 

48  DATA  STR6/' LOWER  PERFORMANCE  SPECIFICATION'/ 

49  DATA  STRl/'MEAN  OF  MEASUREMENT  ERROR'/ 

50  DATA  STR2/ ' STANDARD  DEVIATION  OF  MEASUREMENT  ERROR'/ 

51  DATA  STR3/'MEAN  OF  TRUE  PERFORMANCE'/ 
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52  DATA  STR4/ ' STANDARD  DEVIATION  OF  TRUE  PERFORMANCE'/ 

53  DATA  STR7/'  UPPER  TOLERANCE  FACTOR  OF  ATE'/ 

54  DATA  STR8/' LOWER  TOLERANCE  FACTOR  OF  ATE*/ 

55  C 

56  C    INPUT  COMMON  VALUES. 

57  C 

58  CALL  INPUT  (MEANE ,STR1) 

59  CALL  INPUT  (SDE,STR2) 

60  CALL  INPUT  (MEANU, STR3) 

61  CALL  INPUT  (SDU,STR4) 

62  C 

63  C    DETERMINE  WHAT  TYPE  OF  TEST  IS  DONE  AND  CALL  APPROPRIATE 

64  C    SUBROUTINE.    UPPER-TAIL  MEANS  ONLY  TRUE  PERFORMANCE  BELOW 

65  C    SOME  VALUE  X  IS  ACCEPTABLE .    LOWER-TAIL  MEANS  ONLY  PERFORMANCE 
65  C    ABOVE  VALUE  Y  IS  ACCEPTABLE.    TWO-TAIL  MEANS  ONLY  PERFORMANCE 

67  C    BETWEEN  VALUES  X  AND  Y  IS  ACCEPTABLE. 

68  C 

69  WRITE(6,2000) 

70  2000      FORMATC   IS  THIS  A  TWO-TAIL  TEST  OF  PERFORMANCE?( Y/N) ' ) 

71  READ(5,1000)  YESNO 

72  1000  FORMAT(Al) 

73  IF  (YESNO. EQ. 'Y' )  CALL  CLPL2(MEANE ,SDE, MEANU, SDU,STR1,STR2, 

74  +  STR3,STR4,STR5,STR6,STR7,STR8) 

75  WRITE(6,2010) 

76  2010     FORMAT ( 1  IS  THIS  AN  UPPER-TAIL  TEST? (Y/N) 1 ) 

77  READ(5,1000)  YESNO 

78  IF  (YESNO. EQ. 1 Y1 )  CALL  CLPLUP(MEANE ,SDE ,MEANU,SDU,STR1,STR2, 

79  +  STR3  STR4  STR5  STR7) 

80  IF  (YESNo'.EQ. 1 N'  )  CALL  CLPLOW( MEANE ,SDE , MEANU, SDU, STR1  ,STR2, 

81  +  STR3,STR4,STR6,STR8) 

82  1  STOP 

83  END 

84  C 

85  C 

86  C 

87  SUBROUTINE  CLPL2(ME,STE,MU,STU,S1,S2,S3,S4,S5,S6,S7,S8) 

88  EXTERNAL  MDNOR,MDBNOR 

89  C 

90  C    MDNOR  &  MDBNOR  ARE  IMSL  ROUTINES.    SEE  IMSL  MANUAL  FOR  INFO.. 

91  C 

92  REAL  SU,SL,ME,MU,STE,STU,BU,BL,RH0,K1,K2,K3,K4,P1,P2,P3,P4 

93  REAL  Ql, 02,03, Q4.CL, PL 

94  INTEGER  IER 

95  CHARACTER*40  S1,S2,S3,S4,S5,S6,S7,S8 

96  CHARACTER"*!  YN 

97  C 

98  C    INPUT  UPPER  AND  LOWER  LIMITS  AND  TOLERANCE  FACTORS. 

99  C 

100  CALL  INPUT(SU,S5) 

101  CALL  INPHT(SL,S6) 

102  CALL  INPUT(BU,S7) 

103  CALL  INPIJT(BL,S8) 

104  C 

105  C    CALCULATE  NEEDED  VALUES.    RH0/STU=1/SQRT( (STU**2)+(STE**2) ) 

106  C 

107  2  RHO=STU/SQRT((STU*STU)+(STE*STE)) 

108  K1=(SU-MU)/STU 

109  K2=(MU-SL)/STU 

110  K3=-K1 

111  K4=-K2 
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112  Q1=(SU-MU-(3U*STE)-ME)*RH0/STU 

113  Q2=(MU-SL-(BL*STE)+ME)*RH0/STU 

114  Q3=-Q1 

115  Q4=-Q2 

116  CALL  MDBN0R(K3,Q2,RH0,P15IER) 

117  CALL  MDBN0R(K3,Q3,RH0,P2,IER) 

118  CALL  MDBN0R(K4,Q4,RH0,P3,IER) 

119  CALL  MDBN0R(K4,Q1,RH0,P4,IER) 

120  CL=P1-P2-P3+P4 

121  CALL  MDN0R(K1,P1) 

122  CALL  MDN0R(K2,P2) 

123  CALL  MDN0R(Q1,P3) 

124  CALL  MDN0R(Q2,P4) 

125  PL=CL+P1+P2-P3-P4 

126  C 

127  C    WRITE  RESULTS. 

128  C 

129  WRITE(6,2100)S1,ME,S2,STE,S3,MU,S4,STU,S5,SU,S6,SL,S7,BU, 

130  +  S8,BL,CL,PL 

131  2100     FORMAT ( 1  FOR: 1 ,/,8( 1X,A40, '    =    1 ,G15.8,/)/,5X, 'THEN  \ 

132  +  1  CONSUMER  LOSS  =         \F10,6,/, 5X , '  AMD  PRODUCER  LOSS' 

133  +  '  =  ',F10.6) 

134  C 

135  C    OPPORTUNITY  FOR  SENSITIVITY  ANALYSIS. 

136  C 

137  WRITE(6,2U0) 

138  2110     FORMAT ( 1  WOULO  YOU  LIKE  TO  00  A  SENSITIVITY  ANALYSIS?( Y/N) 1 ) 

139  READ(5,1100)YN 

140  1100  FORMAT(Al) 

141  IF  (YN  .NE.'Y')  STOP 

142  WRITE(6,2120)  S1,S2,S7,S8 

143  2120     FORMAT ( 1  ENTER  NEW  VALUES  FOR  ' ,A40, ' , ' ,/,lX,A40, '  ,\A40, 

144  +       AND' ,/,lX,A40,\  ONE  VALUE  PER  PROMPT.') 

145  READ  *,  ME,STE,BU,BL 

146  GO  TO  2 

147  END 

148  C 

149  C 

150  C 

151  SUBROUTINE  CLPLUP  (ME,STE,MU,STU,S1,S2,S3,S4,S5,S7) 

152  EXTERNAL  MDNOR.MDB NOR 

153  REAL  ME , STE ,MU , STU ,BU , SU ,RHO ,  Kl , K3 , PI , P2 , Ql , 03 , CL , PL 

154  INTEGER  IER 

155  CHARACTER*40  S1,S2,S3,S4,S5,S7 

156  CHARACTER*1  YN 

157  C 

158  C    INPUT  UPPER  LIMIT  AND  TOLERANCE  FACTOR. 

159  C 

160  CALL  INPUT  (SU.S5) 

161  CALL  INPUT(BU,S7) 

162  C 

163  C    CALCULATE  NEEDEO  VALUES. 

164  C 

165  3  RHO=STU/SQRT((STU*STU)+(STE*STE)) 

166  K1=(SU-M!I)/STU 

167  K3=-K1 

168  Ql= ( SU-MU- ( BU*STE ) -ME ) *RHO/STU 

169  Q3=-Q1 

170  CALL  MDBM0R(K3,Q3,RH0,P1,IER) 

171  CALL  MDN0R(K3,P2) 
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CL=P2-P1 

CALL  MDN0R(Q3,P2) 

174  PL=P2-P1 

175  C 

176  C    WRITE  RESULTS. 

177  C 

178  WRITE(6,2200)  S1,ME,S2,STE,S3,MU,S4,STU,S5, SU,S7,BU,CL,PL 

179  2200     FORMATC  FOR: 1 ,/,6(lX,A40, 1    =    ' ,G15.8,/)/,5X, 'THEN 

180  +  'CONSUMER  LOSS=         ' ,F10. 6,/, 5X, 'AND  PRODUCER  LOSS= 

181  +  F10.6) 

182  C 

183  C    OPPORTUNITY  FOR  SENSITIVITY  ANALYSIS. 

184  C 

185  WRITE(6,2210) 

186  2210     FORMATC  WOULD  YOU  LIKE  TO  00  A  SENSITIVITY  ANALYSIS?( Y/N) ' ) 

187  READ(5,1200)YN 

188  1200  FORMAT(Al) 

189  IF(YN.NE. ' Y' )  STOP 

190  WRITE(6,2220)  S1,S2,S7 

191  2220     FORMATC  ENTER  NEW  VALUES  FOR  ' ,A40, ',',/, IX, A40, ', AMD  ', 

192  +  A40,',',/,'  ONE  VALUE  PER  PROMPT.') 

193  READ  *,  ME,STE,BU 

194  GO  TO  3 

195  END 

196  C 

197  C 

198  C 

199  SUBROUTINE  CLPL0W(ME,STE,MU,STU,S1 ,S2,S3,S4,S6,S8) 

200  EXTERNAL  MDNOR.MDBNOR 

201  REAL  ME,STE,MU,STU,BL,SL,RH0,K2,K4,Q2,Q4,P1,P2,CL,PL 

202  INTEGER  IER 

203  CHARACTERMO  S1,S2,S3,S4,S6,S8 

204  CHARACTERS  YN 

205  C 

206  C    INPUT  LOWER  LIMIT  AND  TOLERANCE  FACTOR. 

207  C 

208  CALL  INPUT(SL,S6) 

209  CALL  INPUT(BL,S8) 

210  C 

211  C    CALCULATE  NEEDED  VALUES. 

212  C 

213  4  RHO=STU/SQRT((STU*STU)+(STE*STE)) 

214  K2=(MU-SL)/STU 

215  K4=-K2 

216  Q2=(MU-SL-(BL*STE)+ME)*RH0/STU 

217  Q4=-Q2 

218  CALL  MDBN0R(K4,Q4,RH0,P1,IER) 

219  CALL  MDN0R(K4,P2) 

220  CL=P2-P1 

221  CALL  MDN0R(Q4,P2) 

222  PL=P2-P1 

223  C 

224  C    WRITE  RESULTS. 

225  C 

226  WRITE(6,2350)  S1,ME,S2,STE,S3,MU,S4,STU,S6,SL,S8,BL,CL,PL 

227  2350     FORMATC  FOR: ' ,/,6(lX,A40, '    =    ' ,G15.8,/)/,5X, 'THEN  ', 

228  +  'CONSUMER  LOSS=         ' ,F10. 6,/, 5X, 'AND  PRODUCER  LOSS  = 

229  +  F10.6) 

230  C 

231  C    OPPORTUNITY  FOR  SENSITIVITY  ANALYSIS. 
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232 

C 

233 

WRITE(5,2310) 

234 

2310 

FORMAT ( 1  WOULD  YOU  LIKE  TO  00  A  SENSITIVITY  ANALYSIS?( Y/N) 1 ) 

235 

REA0(5,1350)  YN 

236 

1350 

FORMAT(Al) 

237 

IF  (YN  . NE.  'Y')  STOP 

238 

WRITE(6,2320)  S1,S2SSR 

239 

2320 

FORMATC  ENTER  NEW  VALUES  FOR  ' ,A40, ',',/, IX, A40, ' , AND  ', 

240 

+  A40, ',',/,'  ONE  VALUE  PER  PROMPT. ' ) 

241 

READ  *,  ME.STE.8L 

242 

GO  TO  4 

243 

END 

244 

C 

245 

C 

246 

C 

247 

SUBROUTINE  INPUT  (VAR, STRING) 

248 

REAL  VAR 

249 

CHARACf ER*40  STRING 

250 

WRITE(6,4000)  STRING 

251 

4000 

FORMATC  ENTER  1  ,A40) 

252 

READ  *,VAR 

253 

END 

END  PRT 
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Appendix  B.  Plots  of  Consumer's  Loss  and  Producer's  Loss  as  Functions  of 
the  Mean  and  Standard  Deviation  of  Measurement  Error 


B.l  Introduction 

This  appendix  presents  plots  illustrating  the  effects  of  changes  in  the 
mean  and  standard  deviation  of  the  measurement  error  of  test  equipment  on 
Consumer's  Loss  (CL)  and  Producer's  Loss  (PL)  when  the  distribution  of  UUT 
performance  and  the  performance  specifications  are  both  held  constant.  These 
effects  are  shown  for  a  range  of  alternative  values  of  test  specification 
tolerances.    Examples  are  given  of  how  to  read  and  interpret  the  plots.  These 
plots  have  the  same  general  applications  as  the  computer  program  which 
generated  them,  BINORMAL.CL-PL;  however,  they  are  somewhat  less  flexible  than 
the  program  because  the  range  of  situations  they  cover  is  limited. 
Consequently,  the  graphs  are  most  useful  when  approximations  of  CL  and  PL  are 
adequate  or  when  access  to  a  computer  that  has  BINORMAL.CL-PL  installed  is 
unavailable  or  limited.     The  algorithms  used  tc  calculate  CL  and  PL  are  not 
discussed  in  this  appendix,  but  are  given  in  appendix  A  and  explained  in  more 
depth  in  section  2.     The  notation  used  in  this  appendix  is  the  same  as  used  in 
the  main  body  of  the  report  and  as  defined  in  table  2.1. 
B.2  Scope 

Steps  were  taken  to  ensure  that  the  values  used  to  generate  CL  and  PL 
were  reasonable  and  representative  of  values  found  in  actual  test  equipment 
applications;  however,  because  of  space  limitations  not  all  conceivable 
applications  are  represented.     For  instance,  the  graphs  show  CL  and  PL  for 
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two-tail  tests,  but  many  applications  call  for  one-tail  tests.     Similarly,  all 
systematic  measurement  errors  and  tolerances  used  in  these  plots  are 
non-negative,  although  these  parameters  can  certainly  assume  negative  values 
in  actual  test  situations. 

There  are  two  restrictions  common  to  all  the  figures  in  this  appendix. 
The  first  is  that  the  underlying  UUT  performance  distribution  is  standard 
normal  (i.e.,  uu  ■  0,  au  =  1).     The  second  restriction  is  that  the  test  is 
two-tailed  with  performance  specifications  and  test  specifications  symmetric 
around  yu  and  performance  specifications  set  at  Sy  =  -S^  =  2ou.     It  is 
possible  to  make  adjustments  to  any  data  set  for  which  the  UUT  performance  can 
be  treated  as  normal  in  order  to  satisfy  the  first  restriction.  The  second 
restriction,  however,  must  be  strictly  satisfied.    To  illustrate  how  these 
restrictions  affect  the  applicability  of  the  figures,  suppose  the  following 
parameters  describe  a  test  situation:     mu  =  10,  ou=  2,  ye  =  4,  oe  =  1 ,  S\j  = 
14,  SL  =  6,  and  TA  =  4.    The  first  step  in  adjusting  the  data  to  satisfy  the 
first  restriction  is  to  standardize  the  UUT  performance  specifications  and  the 
mean  UUT  performance.    Denoting  the  resulting  adjusted  parameters  with  an 
asterisk,  we  have  uu*  =  (10-10)/2  =  0,  Sv*  =  (14-10)/2  =  2,  and  SL*=(6-10)/2  = 
-2,     Because  Sy*  ■  ""S^*  ■  2  in  this  case  and  because  the  test  specifications 
are  symmetric  at  TA5S4,  the  second  restriction  is  satisfied  and  it  is  possible 
to  use  the  plots  to  analyze  the  situation.     If  this  were  not  the  case, 
BINORMAL.CL-PL  would  have  to  be  used  for  the  analysis.    The  next  step  in 
adjusting  the  parameters  is  to  divide  all  remaining  values  by  0U  to  convert 
them  to  the  new  scale.     Thus,  ou*  =  2/2  =  1,  ae*  =  1/2  =  .5,  ue*  «  4/2  =  2, 
and  TA*  ■  4/2  ■  2.    These  values  should  be  used  to  identify  which  plots  are 
appropriate  for  analyzing  the  situation. 
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B.3    Interpretation  of  the  Plots 

There  are  three  figures  showing  the  effect  of  systematic  measurement 
error  on  CL  (figures  B.l,  B.2,  and  B.3),  and  three  showing  its  effect  on  PL 
(figures  B.4,  B.5,  and  B.6).     The  assumed  standard  deviation  of  the 
measurement  error,  a£ ,  is  0.5  in  figures  B.l  and  B.4,  0.25  in  figures  B.2  and 
B.5,  and  0.1  in  figures  B.3  and  B.6.     Other  than  this  variation,  all 
assumptions  are  the  same  for  these  six  figures.    There  are  also  three  figures 
showing  the  effect  of  the  standard  deviation  of  the  measurement  error  on  CL 
(figures  B.7,  B.8,  and  B.9)  and  three  showing  its  effect  on  PL  (figures  B.10, 
B.ll,  and  B.12).    The  assumed  systematic  measurement  error  is  0.5  in  figures 
B.7  and  B.10,  0.25  in  figures  B.8  and  B.ll,  and  0.0  in  figures  B.9  and  B.12. 
Other  than  this  variation,  all  assumptions  for  these  six  figures  are  the  same. 

For  all  twelve  figures,  several  plots  are  given,  each  plot  having  a 
different  assumed  level  of  TA.    This  arrangement  permits  one  to  observe  what 
effect  TA  has  on  CL  and  PL  for  the  different  situations.    As  was  discussed  in 
section  3,  the  value  of  TA  has  significant  effects,  but  in  opposite 
directions,  on  CL  and  PL.    As  can  be  observed  in  all  the  figures,  a  decrease 
in  TA  increases  CL  but  decreases  PL. 

Note  that  the  figures  showing  the  effects  of  ae  (i.e.,  figures  B.7 
through  B.12)  do  not  all  have  the  same  number  of  plots.    As  stated  in 
expression  (3.8)  of  section  3,  when  TA  is  smaller  than  the  systematic 
measurement  error,  a  decrease  in  oe  does  not  always  cause  a  decrease  in  CL. 
The  plots  included  in  the  figures  are  restricted  to  those  that  represent 
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situations  where  it  is  clearly  advantageous  to  decrease  ye  and  ae.l 
Consequently,  no  plots  were  included  in  figures  B.7  through  B.9  to  show  the 
effect  of  a£  on  CL  for  cases  in  which  TA  is  smaller  than  ye  .     Figures  B.10 
through  B.12  show  the  effect  of  o£  on  PL  for  the  same  TA  values  used  in 
figures  B.7  through  B.9. 


For  a  discussion  of  this  phenomenon  and  more  information  about  the  effect  of 
ye  and  ae  on  CL  and  PL  in  situations  not  covered  by  the  figures,  see  section 
3. 


66 


o.oae  o.25ae  o.50ae  o.75ae  i.ooae  i.25ae  i.50ae  U5ae  2.ooae 
SYSTEMATIC  MEASUREMENT  ERROR  (BIAS) ,  Me 

Figure  B.l.    Effect  of  Systematic  Measurement  Error  on  Consumer's  Loss 
(oe=0.5,  yu-0,  ou=l,  Su=2,  SL=-2) 
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SYSTEMATIC  MEASUREMENT  ERROR  (BIAS),/x6 


Figure  B.2.     Effect  of  Systematic  Measurement  Error  on  Consumer's  Loss 
(o£=0.25,  Mu=0,  au=l,  Su=2,  SL— 2)  67 
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o.oae  0.25 ae  o.50ae  0.75 ae  i.oo<7e  i.2sae  i.5oae   1.75 af  2.ooaf 
SYSTEMATIC  MEASUREMENT  ERROR  (BIAS), Me 

Figure  B.3.     Effect  of  Systematic  Measurement  Error  on  Consumer's  Loss 
(oe-0.1,  yu=0,  ou=l,  Su=2,  SL=-2) 
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0.0<7e  O.250e  0.50Oe  0.75O"e   1.00(7e   1.25ae   ).50ae   175ae  2.ooae 
SYSTEMATIC  MEASUREMENT  ERROR  (BIAS) ,  fx€ 


Figure  B.4.     Effect  of  Systematic  Measurement  Error  on  Producer's  Loss 
(0e=O.5,  uu=0,  ou«l,  Su=2,  SL=-2) 
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SYSTEMATIC  MEASUREMENT  ERROR  (BIAS),^€ 

Figure  B.5.     Effect  of  Systematic  Measurement  Error  on  Producer's  Loss 
(oe=0.25,  uu=0,  ou=l,  Su=2,  SL— 2) 
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SYSTEMATIC  MEASUREMENT  ERROR  (BIAS),  jue 

Figure  B.6.     Effect  of  Systematic  Measurement  Error  on  Producer's  Loss 
(oe=G.l,  yu=0,  ou=l,  S„=2,  SL=-2) 
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B.4     Illustrative  Applications  of  the  Figures 

In  this  subsection  several  examples  are  presented  to  illustrate  how  one 
can  apply  the  figures  to  actual  test  situations.     Suppose  ATE  is  used  in  an 
electronics  manufacturing  plant  to  test  a  resistor  that  is  a  critical  part  of 
a  calculator  produced  there.     The  resistor  is  known  to  have  a  normal 
performance  distribution  with  mean  100.0  ohms  and  standard  deviation  0.4  ohm. 
Historical  records  show  that  if  the  resistor  performance  is  not  within  two 
standard  deviations  of  the  mean,  then  the  calculator  is  very  likely  to  fail 
during  the  one-year  warranty  period.     On  the  basis  of  these  records,  the 
limits  of  acceptable  performance  for  this  resistor  have  been  set  at  yu±2ou 
ohms.     The  plant  quality  assurance  engineers  also  know  that,  for  this  range  of 
resistance  values  and  for  the  current  system  and  frequency  of  calibration,  the 
ATE  readings  are  normally  distributed  with  a  typical  systematic  measurement 
error  equal  to  0.1  ohm  and  standard  deviation  of  the  measurement  error  equal 
to  0.1  ohm.     In  the  past,  the  engineers  have  always  set  the  test  tolerance  at 
0.1  ohm  to  adjust  for  the  measurement  errors,  but  the  resulting  PL  has  been 
determined  to  be  too  high  (i.e.,  the  yield  is  too  low  because  of  unnecessary 
rejections).     The  plant  managers  want  the  engineers  to  find  out  what  can  be 
done  to  decrease  PL  and  how  CL  will  be  affected  as  a  result. 

Case  I;     The  first  step  in  applying  the  figures  to  analyze  this  problem  is  to 
make  sure  the  restrictions  discussed  in  section  B.2  are  satisfied.     This  means 
adjusting  the  parameters  as  described  in  section  B.2  and  observing  whether  Sy* 
=  -Sl*  =  2.     The  values  given  for  the  parameters  were  uu  =»  100.0,  au  ■  0.4,  Su 
=  yu+2au  =  100.8,     SL  -  Uu-2au  *  99.2,  ue  -0.1,  oe  =0.1,  and  TA  -  0.1. 
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Standardizing  uu,  Sjj,  and  Sl,  one  obtains  yu*  =  (100.0-100.0)/0.4  =  0,  Sy*  = 
(100.8-100.0)/0.4  =  2.0,  and  Sl*  =  (99.2-100.0)/0.4  =  -2.0.     Because  Su*  = 
-Sl*  =  2,  the  figures  can  be  used  to  analyze  this  problem.     Dividing  the  rest 
of  the  values  by  ou  to  convert  to  the  new  scale  yields  ou*  =  0.4/0.4  =  1.0, 
ue*  -  0.1/0.4  -  0.25,  oe*  =  0.1/0.4  =  0.25,  and  TA*  =  0.1/0.4  =  0.25.  Since 
both  ue*  and  ae*  equal  0.25,  the  relevant  plots  are  in  figures  B.2,  B.5,  B.8, 
and  B.ll.     Reading  from  figures  B.2  and  B.5  we  observe  CL  =  0.0041  and  PL  = 
0.058  when  ye*  -  0.25  (=1.0a£*)  and  TA*  =  0.25  (=1.0ae*).     Figures  B.8  and 
B.ll  give  the  same  values,  as  expected.     Table  B.l  summarizes  the  parameter 
values  used  and  the  resulting  values  of  CL  and  PL  for  this  base  case  and  all 
remaining  case  illustrations  in  this  section. 

Case  II:     There  are  three  variables  that  could  possibly  be  changed:    ye ,  oe , 
and  TA.     Suppose  improved  or  more  frequent  calibration  of  the  ATE  completely 
eliminated  the  systematic  measurement  error  (i.e.,  Me*=0).     From  figures  B.2 
and  B.5  it  is  seen  that  if  TA  is  held  constant,  then  CL  will  fall  to  0.0018 
and  PL  to  0.046.     Elimination  of  the  systematic  error  causes  a  0.0023 
reduction  in  CL  and  a  0.012  reduction  in  PL.     Thus,  out  of  every  lot  of  10,000 
resistors  tested,  23  fewer  bad  UUTs  will  be  incorrectly  accepted  and  120  fewer 
good  UUTs  will  be  unnecessarily  rejected. 

Case  III;     Now  suppose  that  instead  of  improving  ATE  calibration,  the  company 
buys  more  precise  ATE,  which  reduces  o£  to  0.04  ohm.     Then  oe*  ■  0.1  (= 
0.1ou*)  and  it  is  seen  from  figures  B.8  and  B.ll  that  CL  falls  to  0.0019  and 
PL  falls  to  0.0475.    The  drop  in  oe  causes  a  0.0022  reduction  in  CL  and  a 
0.0105  reduction  in  PL.     Thus,  out  of  every  lot  of  10,000  resistors  tested,  22 
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Table  B.l     Parameter  Values  and  Resulting  Values  of  CL  and  PL  for  Case  Examples. 


Parameter 

Parameter  ' 

Values3 

Resulting  Values 

Change 

from  Base 

Case 

Changed 

oe* 

TA* 

CL 

PL 

CL 

PL 

I 

Base 

0.25 

0.25 

0.250 

0.0041 

0.0580 

II 

0.00 

0.25 

0.250 

0.0018 

0.0460 

-0.0023 

-0.0120 

III 

0.25 

0.10 

0.250 

0.0019 

0.0475 

-0.0022 

-0.0105 

IV 

TA 

0.25 

0.25 

0.125 

0.0070 

0.0385 

+0.0029 

-0.0195 

V 

He  »?£ »TA 

0.00 

0.10 

0.050 

0.0019 

0.0088 

-0.0022 

-0.0492 

VI 

ye,TA 

0.00 

0.25 

0.125 

0.0041 

0.0276 

0.0000 

-0.0304 

aSo  that  the  figures  can  be  applied  to  these  case  examples,  each  parameter  value  is  given 
in  units  of  the  standard  deviation  of  the  UUT  distribution.     That  is,  as  indicated  by 
the  asterisk,  each  has  been  divided  by  ou  =  0.4  ohm.     Note  further  that  to  interpret  the 
figures  correctly  the  values  of  ye*  and  TA*  must  both  be  read  in  units  of  <Je*. 
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fewer  bad  UUTs  will  now  be  incorrectly  accepted  and  105  fewer  good  UUTs  will 
now  be  unnecessarily  rejected.     Clearly,  in  this  case  reducing  ue   is  more 
advantageous  than  reducing  o£ .     Of  course,  tbe  decision  between  improving 
calibration  procedures  to  remove  systematic  error  or  purchasing  more  precise 
ATE  would  also  have  to  take  account  of  the  costs  of  implementing  each  policy. 
Case  IV:     Suppose  no  changes  are  made  in  the  measurement  errors,  but  the  test 
specifications  are  loosened  by  decreasing  TA  to  0.05  ohm  (TA*  =  0.125ou*  = 
0.5oe*).     From  figures  B.2  and  B.5,  it  is  seen  that  this  reduces  PL  to  0.0385 
but  increases  CL  to  0.0070.     The  decrease  in  TA  causes  a  0.0195  reduction  in 
PL  and  a  0.0029  increase  in  CL.     The  benefits  of  the  large  decrease  in  PL 
would  have  to  be  compared  to  the  disadvantages  of  the  increase  in  CL  in  order 
for  the  company  to  determine  whether  these  changes  in  CL  and  PL,  considered 
together,  are  indeed  advantageous. 

Case  V:     Changes  in  ue ,  ae  ,  and  TA  can  also  occur  in  combination.  For 
instance,  suppose  the  bias  is  eliminated,  o£  is  reduced  to  0.04  ohm  (oe*  = 
0.1au*),  and  TA  is  reduced  to  0.02  ohm  (TA*  =  0.05au*  =  0.5ae*).     Then  it  can 
be  seen  from  the  TA  =  0.5ce  plots  of  figures  B.3  and  B.6  (where  ae*  =  0.1) 
that  CL  falls  to  0.0019  and  PL  falls  to  0.0088.     The  combined  changes  in  ye , 
oe,  and  TA  cause  a  0.0022  decrease  in  CL  and  a  0.0492  decrease  in  PL  from  the 
original  values.     Thus,  out  of  every  lot  of  10,000  resistors  tested,  22  fewer 
bad  UUTs  will  be  incorrectly  accepted  and  492  fewer  good  UUTs  will  be 
unnecessarily  rejected.     The  only  combination  of  changes  that  cannot  be 
analyzed  with  these  figures  is  a  decrease  in  a£  combined  with  a  decrease  in  TA 
to  a  value  smaller  than  the  assumed  level  of  ue .     This  combination  represents 
the  situation  discussed  in  section  B.3.     It  should  also  be  noted  that  not  all 
possible  values  for  the  three  variables  are  represented,  so  interpolation  must 
sometimes  be  used  to  estimate  the  correct  result. 
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Case  VI;     Another  application  of  these  figures  involves  converting  the  effect 
of  changes  in  ue  or  ae  into  a  single  type  of  loss,  holding  the  other  loss 
value  constant.     This  conversion  would  be  useful  when,  for  example,  a  purchase 
contract  includes  a  specification  limiting  the  allowable  size  of  CL  for  a 
product,  and  the  supplier  wishes  to  hold  CL  at  that  limit  while  reducing  PL  as 
much  as  possible.     It  is  also  advantageous  to  convert  a  change  in  CL  into  an 
equivalent  change  in  PL  whenever  it  is  difficult  or  impossible  to  establish  a 
dollar  value  for  a  one-unit  reduction  in  CL  (e.g.,  in  the  case  of  health  and 
safety  hazards).    To  illustrate  this  application,  consider  again  the  resistor 
example.    When  the  systematic  measurement  error  was  eliminated,  it  was 
determined  that  CL  fell  by  0.0023  and  PL  by  0.012.     But  suppose  the  company 
management  wished  to  keep  CL  at  or  near  the  original  value  of  0.0041.  From 
figure  B.2  it  is  seen  that  this  can  be  accomplished  by  lowering  TA  to  0.05  ohm 
(TA*  =  0.125  =  0.5oe*)  when  the  systematic  error  is  eliminated  (i.e.,  ye  ■  0). 
It  is  seen  from  figure  B.5  that  PL  falls  to  0.0276  when  the  systematic  error 
is  eliminated  and  TA  is  reduced  just  enough  to  keep  CL  at  its  original  level. 
Of  this  total  reduction  in  PL,  0.012  is  directly  due  to  the  elimination  of  the 
systematic  error,  and  the  additional  0.0184  can  be  attributed  to  the  effect  of 
reducing  TA  enough  to  leave  CL  unchanged.     In  this  way,  the  0.0023  reduction 
in  CL  that  is  made  possible  by  eliminating  the  systematic  error  has  been 
converted  to  an  equivalent  0.0184  reduction  in  PL. 

One  can  apply  the  same  method  to  make  CL  as  low  as  possible  while  holding 
PL  constant.  Quality  control  policy  might  call  for  this  type  of  conversion  if 
the  primary  objective  were  to  reduce  reported  failure  rates.    The  extra 
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reduction  in  CL  would  be  accomplished  by  increasing  TA.     In  a  similar  manner, 
it  is  possible  to  achieve  any  desired  ratio  of  CL  to  PL  by  changing  TA.  This 
possibility  exists  even  when  ue  and  oe  both  remain  unchanged.     The  figures 
presented  here  may  require  interpolation  of  TA  in  order  to  estimate  the 
effects  on  CL  and  PL  of  changing  TA.    More  accurate  results  can  be  obtained  by 
using  the  program  B I NORMAL . CL-PL ,  described  in  appendix  A,  and  using  the 
sensitivity  analysis  capability  to  vary  TA  until  acceptably  accurate  results 
are  obtained.    The  ability  to  vary  TA  to  achieve  a  desired  ratio  of  CL  and  PL 
is  useful  in  establishing  the  economically  optimal  trade-off  between  CL  and 
PL,  provided  data  are  available  on  the  relative  cost  of  each  unit  of  CL  and 
PL. 
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supplement  to  the  activities  of  the  private  sector  standardizing 
organizations. 

Consumer  Information  Series — Practical  information,  based  on 
NBS  research  and  experience,  covering  areas  of  interest  to  the  con- 
sumer. Easily  understandable  language  and  illustrations  provide 
useful  background  knowledge  for  shopping  in  today's  tech- 
nological marketplace. 

Order  the  above  NBS  publication!;  from:  Superintendent  of  Docu- 
ments, Government  Printing  Office.  Washington.  DC  20402. 
Order  the  following  NBS  publications — FIPS  and  NBSIR's—from 
the  National  Technical  Information  Service ,  Springfield,  VA  22161 . 

Federal   Information  Processing  Standards  Publications  (FIPS 

PUB) — Publications  in  this  series  collectively  constitute  the 
Federal  Information  Processing  Standards  Register.  The  Register 
serves  as  the  official  source  of  information  in  the  Federal  Govern- 
ment regarding  standards  issued  by  NBS  pursuant  to  the  Federal 
Property  and  Administrative  Services  Act  of  1949  as  amended. 
Public  Law  89-306  (79  Stat.  1127),  and  as  implemented  by  Ex- 
ecutive Order  11717  (38  FR  12315,  dated  May  II.  1973)  and  Part  6 
of  Title  15  CFR  (Code  of  Federal  Regulations). 

NBS  Interagency  Reports  (NBSIR) — A  special  series  of  interim  or 
final  reports  on  work  performed  by  NBS  for  outside  sponsors 
(both  government  and  non-government).  In  general,  initial  dis- 
tribution is  handled  by  the  sponsor;  public  distribution  is  by  the 
National  Technical  Information  Service  ,  Springfield,  VA  22161, 
in  paper  copy  or  microfiche  form. 


U.S.  Department  of  Commerce 

National  Bureau  of  Standards 

Washington,  D.C.  20234 

Official  Business 

Penalty  for  Private  Use  $300 


